COMMENTARY: At a recent IT Symposium, Gartner shared a finding that AI risk has grown faster than AI value. For most CISOs, it’s become an obvious truth of their day-to-day work.
Not only that, but it’s a highly personal one: CISOs are responsible for the risk of AI without managing all of the decisions and workstreams that shape it, putting these leaders into a difficult position. According to IBM, two-thirds of technology chiefs say they are held accountable for AI systems they don’t fully control.
[SC Media Perspectives columns are written by a trusted community of SC Media cybersecurity subject matter experts. Read more Perspectives here.]
It’s easy to blame this risk on agents being unpredictable – they are nondeterministic, after all. But there are operational fixes that can mitigate this risk, and most organizations are not implementing them.
Agents fail at machine speed, but most enterprises still detect and resolve their failures at human speed. This creates a lag between failure and fix that compounds issues, increasing risk and widening the AI trust gap. And it’s often CISOs who have to answer for it.
Every failure has an exposure window
Agent failures don’t typically throw errors, and so one wrong answer doesn’t exist in isolation. An agent that’s wrong often does it with confidence, handing off an incorrect output to a user, a business report, or the next agent in the system where its impact cascades.
Here's a simple example: an agent that answers customer billing questions pulls from a data table that stopped updating over the weekend. Rather than crashing, the agent keeps answering billing questions with stale numbers. If nobody notices until a customer escalates the issue on Monday, that's two days of wrong answers, each one a small hit to customer trust – and possibly a compliance problem.
Worse, the full impact gets felt far beyond the chat window. If downstream workflows – like automated payment reconciliation, CRM logging, or secondary agents handling account adjustments – ingest those erroneous figures, silent data corruption cascades across the core enterprise stack before anyone realizes the source table was stale.
Now picture that kind of failure at enterprise scale. IDC projects that, by 2029, AI agents will carry out more than 217 billion actions every day – roughly 2.5 million every second.
We don’t have to wait for 2029 to see how these volumes amplify risk. Picture a modest deployment today: 200 agents, each handling 2,000 tasks a day. If every task involves around 10 steps, such as model calls, tool calls, and data lookups, that’s 400,000 tasks a day – about 4 million agent actions a day, roughly 2,800 actions a minute.
At that pace, a single four-hour incident spans 67,000 tasks and 670,000 unique actions. Catching it through human review would mean sifting through an enormous amount of telemetry to find what’s essentially a needle in the haystack.
Therein lies the mathematical impossibility: humans can never monitor, investigate, and resolve every agent failure at scale.
Why human review doesn't close the gap
The natural response to AI risk is to add a human-in-the-loop. While human controls matter, our mathematical exercise above quickly revealed how this approach doesn’t scale with agents. What’s worse, it can actually create a false sense of safety.
Our team ran a survey with CDO Magazine earlier this year and found that 62% of senior data and AI leaders rely on human review before release to verify AI outputs. Only 25% said they use automated monitoring. That might work for a handful of agents, but in no way does it work for hundreds or thousands.
What’s more, it can actually introduce greater risk into the system that it’s trying to make safer. IBM’s research found that, in organizations relying on manual governance, incident risk increases as AI adoption scales. On the other hand, those that build control directly into their AI systems experience 25% fewer incidents.
A quick fix isn't the same as a real one
There's another issue with relying on human oversight alone: the fix typically addresses a symptom, not the underlying problem, because it’s the quickest thing to do. And speed becomes critical in the heat of an incident.
When an agent gives a bad answer, the fastest response is usually to adjust the agent, perhaps with a new prompt or an added rule. But many agent failures are a consequence of what the agent tries to retrieve, rather than intrinsic to the agent itself. They might start with the data the agent depends on; a stale table, for example, or a broken pipeline. Perhaps it’s fetching a document that's out of date, or its API credential has expired.
Sixty-four percent of AI practitioners admit that context and data quality has become the part of their system they have the least visibility into, according to our research.
If the root cause sits somewhere in the context layer, yet the fix gets applied to the agent, the risk hasn't gone away. It’s just been hidden, and that’s arguably worse. That same failure will come back to bite later via a different query or through another agent reading the same data.
Truly mitigating risk requires insight across all layers of the stack. Unfortunately depth, like breadth as we explored above, becomes hard to achieve when operating at human speed.
Close the lag
All of this suggests a different way to approach agent risk for CISOs.
If the risk lives in the lag, we need to shrink it. We tend to think of speed as the opposite of safety, but that isn’t quite the case anymore. In the agent era, speed of detection and repair represents one of the most important mechanisms we have to actually ensure safety.
A few practical considerations for CISOs on closing the lag:
Make the exposure window a core metric: When it comes to incident management, most organizations track the number of agent incidents. Few, however, track the length of each one, from how quickly it was detected to how quickly it was fixed – and actually gauging how much happened between those points in time. It’s the same idea as the time-to-detect and time-to-resolve measures IT teams already use, applied to agents. Saying, for example, that the average exposure window has dropped from two hours to 15 minutes is a huge win for AI safety and trust management.
Let AI contain failures, and let people set the rules: No team can sift through millions of agent actions. Detection, triage and diagnosis should run at machine speed across all of the telemetry that agents produce. This includes implementing automatic circuit breakers that slow an agent down, switch it to a simpler fallback process, or stop it from using data that looks wrong. Humans stay in control where it matters: deciding when incidents escalate, reviewing the critical cases, and approving changes to policy. That keeps people accountable for decisions without making them the bottleneck for every review.
Focus on the root cause, not the symptoms: When an agent misbehaves, resist the reflex to patch its instructions. Trace every incident from the agent’s output back through the stack underpinning it, including the data and context the agent retrieves. Doing so lets the team implement a tool that actually addresses the root cause rather than just the symptoms, ensuring that the same failure doesn’t quietly return through another query or another agent.
Although most CISOs would agree that AI risk outpaces AI value today, it doesn’t mean we have to slow down our ambitions. Slowing down isn’t an option for enterprises now, we need AI transformation to stay competitive.
Rather, think of this conflict as a sign that the way we manage agents hasn’t caught up with how they work. We still apply human-centric approaches to actors that are not bound by human limitations.
Every organization running agents will have failures; that’s simply the nature of technology. Mitigating this risk requires teams to operationalize trust at the speed of AI rather than at the speed of humans. Until then, the lag will continue and CISOs will constantly play catch-up.
Barr Moses, co-founder and CEO, Monte Carlo
SC Media Perspectives columns are written by a trusted community of SC Media cybersecurity subject matter experts. Each contribution has a goal of bringing a unique voice to important cybersecurity topics. Content strives to be of the highest quality, objective and non-commercial.
