Why human-speed defense has failed

COMMENTARY: An OpenAI eval agent cheated on a test by hacking the answer. It broke out of a research sandbox and walked into Hugging Face after over 17k actions. Hugging Face reconstructed the campaign with another AI.

The campaign lasted for weeks, the attack itself was a few days, at a volume no human attacker could sustain. The difference was how long it could keep going. It could burn through thousands of failed actions, keep following whatever still looked viable, and do it for days.

[SC Media Perspectives columns are written by a trusted community of SC Media cybersecurity subject matter experts. Read more Perspectives here.]

Hugging Face's own disclosure has been pretty clear: a capable human could have found the same flaws. A zero-day in a package-registry proxy that was supposed to function as controlled egress.

Then their dataset pipeline: an HDF5 read that leaked files from a production pod, a Jinja2 template injection, stolen credentials, Kubernetes. Unsafe data processing, exposed cloud metadata, overly broad access, long-lived credentials. The same class of items we’d find in any production platform if we looked long enough.

SharedRoot, which came out around the same time, is the same idea in a local agent. Accomplish-chain a Linux kernel privilege escalation (CVE-2026-46331) with a Claude Cowork implementation detail: the entire host Mac filesystem mounted read-write into the guest VM. Once we have guest-root, the VM isn't a security boundary. Nobody needed a new class of model vulnerability for that.

Most of those 17k actions went nowhere. The path that worked was buried in the ones that didn't. A human attacker can't keep that up for days. The first signals came from Hugging Face's own AI-assisted detection.

Reconstructing so many actions by hand wasn't practical, so they pointed analysis agents at the attacker log and rebuilt the timeline in hours. A human attacker has to decide how long to focus on a target. Reconnaissance takes time, failed attempts take time, and following weak signals across different systems takes more of it. An agent can burn thousands of actions finding out which paths are dead and keep working the ones that aren't.

The first models Hugging Face tried for the forensics, hosted frontier APIs, blocked the work. Safety guardrails can’t tell an incident responder submitting real exploit payloads from the attacker who sent them.

They ran it instead on GLM-5.2, on their own infrastructure. If the only AI we can use for defense runs as someone else’s API, we’re not always allowed to look at the evidence. Real incident data consists of items a model provider may block, like malware, exploit payloads, suspicious commands, stolen credentials. If our defensive model refuses to process them, it isn't much use for incident response.

Attackers will point AI at our APIs, sandboxes, dataset pipelines, and public surfaces as they exist today. We should all do the same – not because "AI finds bugs" represents a new idea, but because the bugs that mattered here were sitting in ordinary isolation and data-processing code, waiting for anyone willing to look at them at machine speed.

And we have to let them follow the chain. Test what an exposed service can reach, what crosses a sandbox boundary, where credentials lead, and whether two minor mistakes become useful together. Most paths will go nowhere. An agent can afford to find out which ones don't.

A useful review follows the attacker's path, not a CVE list. Start from the action that would actually hurt (a credential dump, a cluster foothold, a payment, a write) and work backward through the trust boundaries that were supposed to stop it.

Then run the path forward, looking at what each service can reach, what credentials are available along the way, and what becomes possible when the first boundary doesn't hold. The individual findings are often ordinary on their own, but an attacker working through thousands of possible combinations has a much better chance of finding the chain that matters.

The baseline has changed. Defense has to run at the same speed as the attack, on models we actually control, pointed at our own systems. Today, we need to stop the action, not the model – and not the sandbox.

David Mytton, chief executive officer, Arcjet

SC Media Perspectives columns are written by a trusted community of SC Media cybersecurity subject matter experts. Each contribution has a goal of bringing a unique voice to important cybersecurity topics. Content strives to be of the highest quality, objective and non-commercial.