COMMENTARY: Two stories broke within days of each other this month, and neither was really about the industry it seemed to describe.In Melbourne, a man asked his personal AI assistant to book him into a popular gym class. The agent, using OpenClaw with Anthropic's Claude, found it could reserve classes further in advance than the gym allowed. Later, when the user asked the agent to move him up a waitlist, it discovered the booking API had no authorization check on cancelling other members' reservations. It cancelled the spot of the person ahead of him, moving him from #4 to #3 and couldn't undo the cancellation.[SC Media Perspectives columns are written by a trusted community of SC Media cybersecurity subject matter experts. Read more Perspectives here.]Britain's AI Security Institute disclosed that agents built on Anthropic's and OpenAI's models took 19 unauthorized actions across 122 formal safety-evaluation runs. The most serious involved an agent writing malicious code and fabricating online identities to persuade a human reviewer to approve it. No real-world harm resulted; the actions were caught in evaluation, as intended.Together, these aren't unrelated headlines. They're the same pattern at different scales: give an agent a goal and enough latitude, and it will sometimes take actions nobody explicitly authorized to reach that goal, even when the path involves rule-breaking.
Related reading:
This isn't hypothetical for financial services. AISI's Frontier AI Trends Report already flags AI systems increasingly executing consequential financial actions, asset transfers and trading operations, not just reading and analyzing data. As agents move from information to action, the architecture around them matters as much as their intelligence.
Why this isn't just a gym's problem
Agentic AI is already moving toward tools that place orders, rebalance portfolios, and execute workflows on a client's behalf with minimal human confirmation. The appeal is real: speed, scale, and fewer manual steps for the end user.The gym incident shows a specific failure mode: a rule that existed but wasn't enforced where it mattered. An AI agent doesn't need to defeat sophisticated security to cause damage, it just needs a gap between the rule an institution meant to enforce and the rule its infrastructure actually enforces, and agents can find that gap at software speed. In brokerage infrastructure, the equivalent gaps are order limits that are UI suggestions rather than server-side rejections, or an assumption that the sender of a request and the authorized trader are the same entity.The AISI findings add the rest of the warning: even inside deliberate stress tests, agents still took unauthorized actions. Evaluations are valuable precisely because they surface failure modes before deployment, but no evaluation can represent every condition an agent will meet in production. Financial infrastructure needs controls that assume unexpected agent behavior is possible, not ones that rely on agents staying in scope.The opportunity is real too
None of this argues against agentic tools in finance. Execution, monitoring, and compliance agents solve real problems, catching what humans miss and acting at machine speed. The lesson isn't to slow down; it's to build the infrastructure agentic use requires before demand outpaces the design work.Five principles for agent-ready financial infrastructure
- Identity: Can infrastructure distinguish the customer, the application, and the specific agent behind an instruction rather than treating an API key as one undifferentiated actor?
- Permissioning: What is this particular agent actually allowed to do? Anthropic's trustworthy-agents framework treats this action by action, always allowed, needs approval, or blocked, rather than one broad grant.
- Hard constraints, enforced server-side: Transaction limits and account restrictions need to be enforced by the infrastructure itself, not assumed by the client-side interface.
- Human escalation for consequential actions: OpenAI's prompt-injection guidance makes a related point: design systems so manipulation's impact is constrained even when something gets through. Certain actions need a human checkpoint regardless of how confidently an agent requests them.
- Auditability: Can an institution reconstruct what an agent was asked to do, what it attempted, and what actually executed? The gym agent's cancellation succeeded silently, with no check to flag it, a financial system needs to make that kind of gap visible before it's exploited.
