COMMENTARY: Most enterprise AI security reviews still start with the model. What data trained it? Can it hallucinate? Is the output filtered? Can someone manipulate the prompt? Those questions matter. But as AI moves from chatbots into copilots and autonomous agents, the model is becoming only one part of the attack surface.
[SC Media Perspectives columns are written by a trusted community of SC Media cybersecurity subject matter experts. Read more Perspectives here.]
The bigger risk is often the architecture built around it: the identities it uses, the systems it can reach, the data it can retrieve, and the actions it is allowed to take.
The model is not the security boundary
A language model that drafts an email is relatively easy to contain. An agent that can read internal documents, call APIs, raise tickets, update records or trigger a workflow is different. At that point, security depends less on whether the model gives a bad answer and more on whether the surrounding platform lets that answer become an unsafe action.
That distinction matters. If a malicious prompt can eventually reset a password, pull a customer record or change a cloud configuration, prompt injection is no longer just a content problem. It becomes an authorization problem.
Related reading:
The same is true for retrieval-augmented generation. Teams often focus on whether the model can leak sensitive information, but the more basic question is whether the user or agent should have been able to retrieve that information in the first place. A vector database does not magically solve access control. If the source permissions are wrong, the model simply becomes a faster way to expose the mistake.
Agents make identity the control plane
This is where many AI programs are going to run into familiar security problems wearing new labels. Agents need credentials. They need API keys, service accounts, OAuth tokens and access to third-party tools. They also need scopes and permissions.
If an agent has broad standing access because it is convenient during a pilot, that access tends to survive into production. The result is a non-human identity with more reach than most employees, operating at machine speed and making decisions based on probabilistic output.
That is not primarily a model-risk issue. It is an identity and privilege-management issue.
Before approving an agentic use case, I would want to know exactly which identity it runs as, what that identity can access, whether permissions are temporary or standing, which tools it can invoke, and where human approval is required. If those answers are vague, the use case is not ready for production, regardless of how well the model performs in testing.
Logging has to capture the decision path
Traditional application logs are also not enough. When an AI system takes an action, incident responders need to reconstruct more than the final API call. They need to know who initiated the request, what context the model retrieved, which tool it selected, what instruction it generated, whether a human approved it, and what actually changed.
Without that chain, an investigation can end with a perfectly valid API request and no clear explanation of why it happened.
This is one reason AI security cannot sit only with a specialist model-risk team. Security architecture, IAM, application security, data security and SOC teams all own part of the control environment.
Move architecture review before the pilot becomes a platform
The practical mistake is waiting until an AI proof of concept succeeds before asking these questions. By then, teams have already selected the model, connected data sources, created service accounts and integrated tools. Security is left reviewing a design that has effectively been decided.
AI architecture review needs to happen earlier. Not as a checklist to slow experimentation, but as a way to establish the boundaries within which experimentation is safe.
For every material AI use case, the architecture should make five things unambiguous: identity, data access, tool permissions, approval boundaries and auditability. Model testing still matters, but it should sit inside that architecture rather than substitute for it.
The industry will continue to improve model guardrails, red teaming and prompt-injection defenses. We should. But enterprises that treat AI security mainly as a model problem risk becoming very good at testing the least privileged component in the system.
The real security boundary is the architecture around the model — and that is where the next generation of AI security programs will either succeed or fail.
