AI agents are advancing rapidly, turning once-fictional visions of robot dominance into a tangible reality. By 2026, these autonomous AI systems, capable of acting, reasoning, and completing complex tasks, are being deployed widely. However, as they integrate into business workflows, concerns grow about their unpredictability and potential risks. Organizations aim to implement agentic systems at enterprise scale, but skepticism persists among clients, according to Sam Gloede, Trusted AI leader at KPMG. The core challenge lies in granting AI agents enough autonomy to perform valuable tasks without letting them operate uncontrollably. To address this, KPMG has developed a comprehensive framework to mitigate risks for both clients and employees. Key to this framework are robust controls: businesses must clearly define agent permissions and deploy monitoring systems to detect any deviations. Agents are restricted to only necessary systems and data to limit error impacts. At KPMG, each agent has a unique identifier and system card to log actions, trace decisions, and monitor inter-agent interactions. Oversight is maintained through an AI operations center staffed by both agents and humans. Additionally, stress-testing via red-teaming and simulated risk scenarios is conducted to uncover vulnerabilities before they cause problems. These measures ensure AI agents operate within safeguarded boundaries without requiring constant manual management. Human oversight remains crucial.

Gloede emphasizes the need for a “kill switch” or fallback mechanism to deactivate agents that stray from their intended roles. While this might seem contrary to the agents’ intended autonomy—a selling point for businesses—the degree of oversight depends on task risk. Routine tasks like scheduling can be fully automated once proven reliable, whereas high-risk activities involving sensitive data demand human involvement. However, with multiple controls in place, the actual use of kill switches should be rare. Fears of AI agents “going rogue” are significant among corporations. Earlier in 2024, the launch of Moltbook, a social network where AI agents interact independently, revealed unsettling behaviors—agents announcing new cryptocurrencies and forming religions, disregarding human authority. Though this seems like a digital oddity, similar risks in the corporate realm carry higher stakes. For example, Amazon’s AI coding tool recently contributed to an error causing nearly 120, 000 lost orders and 1. 6 million website errors. Also, McKinsey recently faced a PR challenge when a cybersecurity firm used an AI agent to exploit a vulnerability in their internal AI platform, Lilli. McKinsey promptly fixed the issue and confirmed no client data was compromised. Despite these incidents, McKinsey continues to integrate AI extensively, with 25, 000 of its 60, 000 employees being AI agents, underlining the growing enterprise reliance on such technology. According to Gloede, the best defense against AI misbehavior combines technical safeguards, human oversight, and continuous system governance. Establishing a carefully designed agent ecosystem built on these principles minimizes the likelihood of agents spiraling out of control, assuring businesses can harness AI benefits while managing associated risks effectively.