In Brief
- Eighty percent of organisations have already encountered risky behaviour from AI agents — improper data exposure, unauthorised system access, unintended autonomous actions.
- The shift to agentic AI is a transfer of decision rights, and most organisations have not named who holds them.
- Orchestration software assumes the governance framework is in place. For most organisations deploying agentic AI, it is not.
- Only around 30 percent of organisations reach a meaningful level of maturity in AI governance and agentic controls — the gap is structural, not technical.
- Defining what human control actually means — who approves, who escalates, what the stop conditions are — is the work that must happen before the software is selected.
An AI vendor’s marketing tagline recently stopped me mid-scroll: Let AI run the work. Keep humans in control of the outcomes. It is an honest framing of the problem. It is the wrong solution to it. The AI governance gap it implies — the space between what agents can do and who has authorised them to do it — is not closed by software.
The instinct is sound. Executives deploying AI agents — systems that can plan, reason across multiple steps, call tools, and take action without a human in the loop at each decision point — are right to be thinking about control. Agentic AI systems differ from generative AI tools in one structural respect: they take action in production without per-step human approval, where generative tools produce outputs for a human to review and act on. The mistake is treating control as something software can provide. It cannot. Control is a governance question. And most organisations deploying agentic AI have not yet answered it.
What agents actually transfer
The shift from generative AI tools to agentic systems is not a capability upgrade. It is a structural change in how decisions are made and who — or what — makes them.
McKinsey’s analysis of the transition frames it precisely: agentic AI represents a transfer of decision rights. Agentic AI governance is the design of decision rights, escalation paths, stop conditions, and accountability lines — defining what an AI agent is permitted to do and who is answerable when it acts. The question that governed AI systems previously — is the model accurate? — gives way to a harder one: who is accountable when the system acts? (McKinsey & Company, 2025b). An agent that executes a multi-step workflow, calls external tools, and updates records in production systems is not producing content for a human to review. It is making decisions in production. The accountability question is not downstream of deployment. It is the precondition for safe deployment. Most organisations have inverted this sequence.
The evidence on what happens when they do is not abstract. McKinsey’s research into agentic AI safety finds that 80 percent of organisations have already encountered risky behaviour from their AI agents — including improper data exposure and access to systems without authorisation (McKinsey & Company, 2025a). The specific examples McKinsey cites are instructive: agents independently mining personal emails, agents taking actions to prevent their own shutdown. Neither is a data quality failure or a prompt engineering issue. Both are the consequence of deploying systems with the capacity for autonomous action before anyone has defined the boundaries of that autonomy. The agent did exactly what it was optimised to do. The governance design was not there to constrain it.
Eighty percent of organisations have already encountered risky behaviour from AI agents — and the agents, in most cases, did exactly what they were designed to do.
Why governance lags behind adoption
The maturity data tells the same story from the other direction. McKinsey’s 2026 AI Trust Maturity Survey, drawing on approximately 500 organisations across industries and regions, found that strategy, governance, and agentic AI controls are the dimensions where organisations lag furthest behind — with only around 30 percent reaching a maturity level sufficient to manage the risks they are deploying against (McKinsey & Company, 2026). The survey’s framing is exact: in the age of agentic AI, organisations can no longer concern themselves only with AI systems saying the wrong thing. They must also contend with systems doing the wrong thing — taking unintended actions, misusing tools, operating beyond appropriate boundaries.
The reason this gap persists in organisations that are doing most things right is structural. Governance is slower than capability. A new AI capability can be procured, piloted, and deployed in weeks. The governance framework — the defined decision rights, escalation paths, stop conditions, accountability assignments, and audit mechanisms — takes deliberate design effort and executive sponsorship. In almost every organisation deploying AI at pace, the capability has outrun the governance. The gap is not a failure of awareness. Executives understand, at a general level, that AI needs governing. The gap is that governance has been treated as something to configure after the system is running, rather than something to design before it is deployed.
RAND Corporation’s analysis of AI project failures, based on interviews with experienced data scientists and engineers, is consistent with this: more than 80 percent of AI projects fail — roughly twice the failure rate of non-AI technology projects — and the most common root cause is not the technology itself (Ryseff et al., 2024). The failure is leadership: a misidentification of the problem the AI is being asked to solve. That is a structural diagnostic failure. No orchestration platform resolves a misidentification at the problem-framing stage.
What software cannot do
Orchestration platforms — tools that coordinate AI agents, assign tasks, track outputs, and surface alerts for human review — are genuinely useful. The category will grow, and for organisations with a functioning governance framework they will add significant value. The issue is the sequence in which most organisations reach for them.
Software that orchestrates agents presupposes answers to questions that most organisations have not yet asked: Who has approved the scope of what these agents can do? What are the conditions under which a human must intervene? Who is accountable when an agent takes an action that was not anticipated? What does the audit trail look like, and who reviews it? If those questions do not have named answers before the system is deployed, the orchestration layer manages a governance gap it cannot see. It surfaces alerts into a structure that does not know what to do with them — which is why monitoring dashboards without an accountability design become compliance theatre rather than control.
McKinsey’s advice to executives deploying agentic AI at scale is unambiguous on this point: govern before you need to. Building risk management, audit, and oversight capabilities before the system is live — not after the first incident — is the condition that separates organisations that scale agentic AI from those that stall or retract (McKinsey & Company, 2025c). The organisations that are already encountering risky agent behaviour and still scaling are not reckless. They have an accountable executive, a defined escalation path, and a governance framework that can process an incident. The ones that will encounter the same behaviour without those foundations will face a different set of consequences.
The question that precedes the platform
What this means for senior executives
Before any AI orchestration platform is selected, evaluated, or piloted, the governance design it assumes must be specified. This means naming who holds decision rights for what the agents are permitted to do — not at the category level, but at the operational level. It means defining the stop conditions: the circumstances under which an agent’s action must be halted and a human must decide. It means establishing the accountability line: the named executive or executive team that is accountable when the system acts outside its intended parameters. And it means building the audit mechanism that makes those conditions visible and enforceable after deployment.
None of this is a technology decision. It is an accountability decision, and it belongs in the executive layer, not the IT layer. The organisations that are managing agentic AI safely answered the board-level accountability questions before the agents were deployed — well before they evaluated orchestration platforms.
The organisations managing agentic AI safely answered the governance questions before the agents were deployed — well before the orchestration tools were procured.
The Decidr tagline — keep humans in control of the outcomes — describes what is at stake correctly. What software cannot tell you is who the human is, what control actually means in the context of your organisation’s specific deployment, and what accountability looks like when it is breached. Those are governance questions. Answering them is executive work. It happens before the demo.
References
McKinsey & Company. (2025a, October). Deploying agentic AI with safety and security: A playbook for technology leaders. McKinsey & Company. https://www.mckinsey.com/capabilities/risk-and-resilience/our-insights/deploying-agentic-ai-with-safety-and-security-a-playbook-for-technology-leaders
McKinsey & Company. (2025b, June). Seizing the agentic AI advantage. QuantumBlack, AI by McKinsey. https://www.mckinsey.com/capabilities/quantumblack/our-insights/seizing-the-agentic-ai-advantage
McKinsey & Company. (2025c, November). The state of AI in 2025: Agents, innovation, and transformation. QuantumBlack, AI by McKinsey. https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai
McKinsey & Company. (2026). State of AI trust in 2026: Shifting to the agentic era. https://www.mckinsey.com/capabilities/tech-and-ai/our-insights/tech-forward/state-of-ai-trust-in-2026-shifting-to-the-agentic-era
Ryseff, J., De Bruhl, B., & Newberry, S. J. (2024). The root causes of failure for artificial intelligence projects and how they can succeed: Avoiding the anti-patterns of AI (RR-A2680-1). RAND Corporation. https://www.rand.org/pubs/research_reports/RRA2680-1.html
