COMMENTARY: AI agents are now a disclosed business risk. Apple's latest annual report warns that AI agents can take actions autonomously, including in systems they may not be authorized to access, and that some consequences, such as deleting data or completing transactions, may be difficult or impossible to reverse.
Apple then added that it may have limited ability to oversee and control third-party artificial intelligence agents and other technologies.
[SC Media Perspectives columns are written by a trusted community of SC Media cybersecurity subject matter experts. Read more Perspectives here.]
Risk factors are written to cover what might go wrong, and they are easy to skim. This one deserves a slow read. It lists, one by one, the failure modes security teams have spent the year documenting:
- Actions not directed, instructed, or authorized.
- Exceeding permissions, defying directions, or resisting instructions.
- Circumventing security controls.
- Agents that conceal actions, hide, withhold, or manipulate information.
None of this is hypothetical. By its founder's account, a coding agent at a small software company found an API token in an unrelated file and deleted the production database, along with its backups, in about nine seconds. OpenAI has said it alerted more than 100 organizations whose systems its research agents touched. Weeks before Apple's filing, Anthropic's IPO filing described its own models reaching third-party systems during testing.
Read closely, Apple's risk factor makes three admissions that apply to every company running agents:
- Agents can't be fully tested in advance: Apple notes that because these systems are nondeterministic, their behavior can be difficult to anticipate, control, test and audit, and errors can compound across multistep workflows. Pre-deployment review still matters, but an agent that passed review on Monday can take a different path on Tuesday with the same instructions.
- Most agents aren't ours to control: The agents inside a typical enterprise come from somewhere else: coding assistants, copilots inside SaaS tools, browser agents, and the personal agents employees connect to work accounts. Many run on a person's own credentials. Responsibility has already moved toward whoever grants that access. Large banks have started blocking outside agents that log in with customers' credentials, and some are updating account terms so customers are liable for what an agent they authorized does.
- Some mistakes can't be undone: Deleted data, completed payments and sent messages don't roll back. Detecting the problem an hour later in a log is forensics.
Put those together and the design question answers itself. If we can't fully test an agent, can't control how it was built, and can't reverse what it does, the control has to sit at the one point we do own: the moment the agent acts inside our environment.
In practice that comes down to five disciplines:
- Know what's running: Keep a live inventory of every agent, on laptops, in pipelines and inside SaaS platforms, along with what each one can reach and whose credentials it uses. We can't govern an agent we haven't found.
- Scope authority to the task: An agent that summarizes tickets doesn't need write access to production. Grant access for the job at hand, and treat an agent that inherits everything its user can do as a finding.
- Gate the irreversible: The ability to delete data, move money, change permissions, and send information outside the company should require an explicit approval, or get blocked outright. A good place to start: Apple's own example list, deleting data and completing transactions.
- Keep evidence that doesn't depend on the agent: Apple's filing names concealment as a risk. Our record of what an agent did should come from where the action happened, tied to who launched it and what it was asked to do, and never from the agent's own account of its work.
- Test agents the way attackers will: Do this with poisoned documents, hostile web pages, and ambiguous instructions, before an incident does it on its own.
Apple's disclosure should prompt some uncomfortable questions in boardrooms everywhere. Such as: Do we know which agents are operating in our systems? Do we know what they're capable of doing? And if one goes rogue, can we stop it before the damage is done?
These aren't questions companies can afford to put off until something goes wrong. AI agents are already operating inside their environments, whether they're ready for them or not.
It’s time to adjust to this shift and get ready.
Dor Sarig, co-founder and Chief Builder, Pillar Security
SC Media Perspectives columns are written by a trusted community of SC Media cybersecurity subject matter experts. Each contribution has a goal of bringing a unique voice to important cybersecurity topics. Content strives to be of the highest quality, objective and non-commercial.
