The control question
Most AI security failures are not product failures — they are gaps in knowing where control applies. A security team that cannot locate the control points cannot close the gaps. The system generates outputs, accesses tools, makes decisions, and takes actions, but without architectural visibility into where constraints can be applied, security becomes reactive detection of harm that has already occurred.
The control question is diagnostic: where in the AI decision chain can preventive measures be enforced, and who owns each control point? Answer this question incorrectly, and the security program addresses symptoms while leaving the architectural gaps that produce them.
Five control points
Authority control: Governs what tools the AI system can access and what scope of action it can take. Failure mode: AI systems take consequential actions no human explicitly authorized, with no clear accountability or audit trail.
Input/prompt control: Governs what instructions the system accepts and what external data it can retrieve to inform responses. Failure mode: Attackers manipulate system behavior through crafted inputs or poisoned retrieval sources.
Output control: Governs validation and constraints applied before the system's responses reach downstream applications or trigger automated actions. Failure mode: Harmful, biased, or factually incorrect outputs propagate to business processes without verification.
Governance control: Governs which AI use cases are approved, what oversight mechanisms apply, and how systems are classified and monitored. Failure mode: AI systems deploy without appropriate risk assessment, oversight, or compliance alignment.
Authenticity/provenance control: Governs verification that AI-generated content is properly identified and that claimed identity or source attribution is accurate. Failure mode: Synthetic content is accepted as authentic, enabling fraud, misinformation, or identity spoofing.
What each gap produces
Authority control gaps produce AI systems that can modify production databases, send external communications, or execute financial transactions without explicit human authorization for each action. [OWASP LLM Top 10 (2025) identifies excessive agency — where AI systems are granted more authority than required to complete their assigned functions — as a top-ranked risk category (LLM06). The risk is framed as an authority control failure: systems with excessive tool access, permissions, or autonomous decision-making scope can take consequential actions that no user explicitly authorized, and that may be irreversible] (Source: owasp.org). When credential scope exceeds task requirements, the blast radius of system compromise or prompt manipulation extends beyond the intended use case.
Input and prompt control gaps create attack vectors where external parties can manipulate AI system behavior by injecting malicious instructions through user inputs, document uploads, or retrieval sources. Organizations discover that their AI systems have leaked sensitive information, violated content policies, or taken actions contrary to their intended function — often with no clear attribution to the manipulation source.
Output control gaps result in AI-generated content reaching customers, partners, or automated business processes without verification of accuracy, appropriateness, or compliance with organizational policies. Legal exposure, regulatory violations, and reputational damage accumulate from AI outputs that contradict established organizational positions or contain harmful content.
Governance control gaps produce AI implementations that bypass standard security review processes, operate without defined ownership, or fail to meet regulatory requirements that apply to their specific use case. [NIST AI RMF (2023) identifies four core functions for AI risk management: Govern, Map, Measure, and Manage. The Govern function — which covers organizational policies, accountability, and oversight — is listed first because NIST frames governance as the precondition for effective risk management: without organizational accountability, measurement and management functions cannot be sustained] (Source: airc.nist.gov). Organizations find themselves unable to demonstrate compliance during audits or to provide required documentation about AI system decision-making processes.
Authenticity and provenance control gaps enable synthetic content to be accepted as legitimate in identity verification, document processing, or content moderation workflows. Organizations process fraudulent claims supported by AI-generated evidence, accept deepfake content in identity verification processes, or distribute misinformation because they cannot distinguish authentic content from synthetic alternatives.
Preventive vs detective
Preventive controls constrain what the system can do; detective controls observe what the system has done. The distinction matters because detective controls identify harm after it occurs, while preventive controls stop harm from occurring.
[CISA's "Guidelines for Secure AI System Development" (November 2023) identifies both preventive controls (secure design, secure development) and detective/responsive controls (secure deployment, secure operation and maintenance) as required elements of a secure AI system lifecycle. The four-area structure explicitly requires both control types — treating detection without prevention as insufficient for secure AI operation] (Source: cisa.gov).
Current AI security tooling skews heavily detective. Organizations deploy monitoring solutions that can identify when AI systems produce harmful outputs, access unauthorized resources, or violate content policies — but these solutions trigger after the harmful action has occurred. A detection-only posture means business impact has already materialized before the security program responds.
Control ownership map
| Control point | What it governs | Failure mode when absent | Control type | Lane that owns it |
|---|---|---|---|---|
| Authority | Tool access, agent delegation, credential scope | AI systems take unauthorized actions with no accountability | Preventive | AGENT |
| Input/prompt | System instruction manipulation, context poisoning, indirect injection via retrieval | Attackers manipulate system behavior through crafted inputs | Preventive | APPSEC |
| Output | Validation before action, downstream trust model | Harmful or incorrect outputs reach business processes | Both | APPSEC |
| Governance | AI system approval, use-case authorization, oversight | Systems deploy without appropriate risk assessment | Preventive | GOV |
| Authenticity/provenance | Synthetic content verification, identity confirmation | Synthetic content accepted as authentic | Detective | FRAUD |
The AGENT lane owns authority controls in depth: tool authorization, agent identity, and scoped credentials. The GOV lane owns governance controls: AI governance program structure, approval workflows, and regulatory compliance. The APPSEC lane owns input and output controls: prompt injection prevention, context manipulation detection, and output validation. The FRAUD lane owns authenticity controls: synthetic content detection and identity verification in AI contexts.
A security team that can locate all five control points and assigns lane ownership to each has a program structure. One that cannot locate them leaves gaps that no tool purchase closes — because the gaps are architectural, not product-shaped.
Sources
- OWASP: https://owasp.org
- NIST (nist.gov, csrc, nvlpubs, airc): https://airc.nist.gov
- CISA Known Exploited Vulnerabilities: https://cisa.gov
