AI security’s cost bottleneck isn’t tokens – it’s validation

A recent report by Axios claims a company accidentally spent $500 million in one month on Claude usage after failing to implement usage limits for employees. This extreme anecdote punctuates growing uncertainty about how token usage and API bills could become a major bottleneck for companies seeking to reap the productivity benefits of AI tools.

Even major tech companies are reportedly seeking to reel in their AI spending, with The Verge reporting that Microsoft is canceling its Claude Code licenses to steer employees toward its own GitHub Copilot and Uber CTO Praveen Neppalli Naga telling The Information the company used up its entire AI coding budget for 2026 within four months.

How does this fit into cybersecurity? With the landmark moment of Anthropic’s Claude Mythos’ release under Project Glasswing, AI-driven code review and vulnerability discovery are gaining interest, but an analysis by Contrast Security offers a sobering look at the “hidden cost of AI security scanners.”

Contrast’s research found that the biggest spend for organizations seeking to use AI to scan their code for vulnerabilities isn’t the API bill, but the cost of triaging and validating thousands of findings, including a huge number of false positives and inconsistent findings between runs and models.

For example, a simple scan of 1.8 million lines of code using Claude Sonnet 4.6 surfaced 3,560 findings and cost just $315 in token usage, but those 3,560 findings don’t triage and validate themselves. Contrast calculated that if a security engineer making $150,000 per year spent half an hour triaging each finding, the labor cost would come out to $128,000.

The labor costs could be lower if teams prioritized only the high-severity findings, but the non-deterministic nature of AI scanners complicates this process. Contrast found that the same scanner run three times on a smaller 50,000-line sample only agreed with its own findings about 17% of the time, and that some of the consistent findings were assigned inconsistent severity scores by the AI between runs.

“You can filter noise, but filtering based on severity ratings that change run-to-run is just picking from a coin flip. You need deterministic tooling that produces consistent results and runtime context so you can actually trust your risk rankings,” Contrast Security CISO David Lindner told SC Media. “Without that consistency, severity is meaningless.”

Contrast also ran Claude Security, based on Claude Opus 4.7, on the same 50,000-line code base and found similar inconsistencies, with only 25% reproducibility across three scans. A multi-model analysis using a panel of Claude Opus 4.7, Google’s Gemini 2.5 Flash and OpenAI’s GPT-5.5 helped narrow 523 of Claude Opus’ findings down, confirming that 14.9% were true positives, but about half of the remaining 57% flagged for review were still false positives. Additionally, the researchers estimated a full multi-agent scan of its 1.8 million-line codebase would cost between $43,000 to $107,000 in API costs.

“The issue is that pure AI scanning is fundamentally non-deterministic. Cloudflare’s own research on Mythos showed the model wanders without clear direction and context. A better model does not fix that. Non-determinism is not a model quality problem, it’s how large language models work,” Lindner said. “Better models still return different results on every run against the same code. You cannot build a repeatable security program on a tool that cannot tell you the same thing twice.”

A blog post by Daniel Stenberg, the creator of curl, lends some more insight into what a Mythos code scan looks like. A scan of the utility’s 176,000 lines of code produced five “confirmed security vulnerabilities,” which the curl team narrowed to just one real low-severity vulnerability after a few hours of analysis, reflecting the continued necessity of human expert review.

What positive ROI looks like for AI cybersecurity

Vulnerability discovery is not the only area where AI is being used in cybersecurity — organizations are increasingly using AI in security operations centers (SOCs) to enhance threat intelligence, streamline patch management and more.

The return-on-investment (ROI) for AI in security will depend on how it’s applied, Eran Shtauber, CEO of ULTRA RED, a KELA group company, told SC Media.

“The best ROI comes from using AI where reasoning creates value and deterministic systems cannot,” Shtauber said.

High ROI areas identified by Shtauber include:

In other areas, such as code review and vulnerability discovery, organizations are more likely to struggle with balancing cost and benefits, as AI produces large volumes of findings that require further investigation.

“AI reduces the cost of generating security work. It does not automatically reduce the cost of proving that work matters. The organizations achieving the best ROI are not replacing deterministic security with AI, but combining the two,” Shtauber said. “Traditional systems provide the reality and AI provides the intelligence.”

Like Contrast Security, Shtauber said “the biggest hidden cost is not tokens, [it’s] validation.”

These challenges do not mean organizations should give up on the idea of using AI to find vulnerabilities: the technology’s unique reasoning capabilities create new opportunities to discover flaws that traditional scanners miss, as demonstrated by Mythos’ discovery of new vulnerabilities in heavily scanned code, including a 27-year-old flaw in OpenBSD and a 16-year-old one in FFmpeg.

Stenberg, in his blog post, noted some of the benefits of AI scanners over traditional tools, including built-in knowledge about open-source code bases, third-party libraries and protocols, the ability to read comments and understand when they contradict code function, the ability to summarize flaws in natural language and offer patch recommendations, and the ability to check code for platforms and configurations that cannot be scanned with traditional tools.

“Not using AI code analyzers in your project means that you leave adversaries and attackers time and opportunity to find and exploit the flaws you don’t fix,” the curl creator wrote.

Contrast Security’s Lindner said defenders are at more of a disadvantage than attackers when it comes to dealing with the inconsistencies in the findings that AI scanning tools produce.

“Attackers don’t care about false positives as long as they find enough true positives to work with. That’s a real problem, especially for public code,” Lindner said.

Achieving real value and ROI from AI-driven code scanning means understanding the technology’s limitations and using it to enhance the abilities of traditional tools and human analysts, not to replace them, Lindner concluded.

“Constrain the scope to new code and high-risk changes instead of full-coverage scans that generate noise at scale. Pair it with deterministic tools (IAST, SAST, etc) to anchor findings to something repeatable, because LLM non-determinism makes risk tracking impossible on its own,” Lindner advised. “Add runtime visibility to know what code is actually being hit and probed in production. That combination, in the dev cycle, is cost-effective. Pure AI scanning at full coverage is not.”

ULTRA RED’s Shtauber also cautioned against forgoing traditional tools for AI and recommended using a combination of AI, deterministic tools and human expertise to complement one another.

“Avoid spending significant AI resources on repetitive tasks that deterministic systems already perform efficiently. Validating before escalating is crucial. One of the biggest challenges in AI security is distinguishing what’s possible, what’s probable, and what’s proven,” Shtauber said. “Organizations should establish workflows where AI-generated hypotheses are validated before becoming operational findings.”

SC Media reached out to Anthropic for comment on this story and did not recieve a response.