COMMENTARY: Much of the recent AI coverage centers on autonomous systems slipping out of human control. Whatever one thinks of those long-term debates, they distract from the more immediate problem security teams face today: humans now hold tools that became very powerful, very fast, and too few of those humans answer for how the tools are being used.I'm not so much worried about a rogue model deciding to attack a power grid. Rather, we should focus on the ability of a nation-state or cyber criminal using AI to accelerate reconnaissance, phishing, and exploit development. We’re also concerned about an organization carelessly connecting an AI agent to production systems with broad permissions and no meaningful oversight.In both cases, the failures are clearly human.[SC Media Perspectives columns are written by a trusted community of SC Media cybersecurity subject matter experts. Read more Perspectives here.]Regulators cannot write detailed technical rules fast enough to govern this. By the time a prescriptive standard gets finalized, the technology it describes has changed, or loopholes are developed to evade regulations.What does keep pace?Strict financial accountability. Executives adjust quickly when the consequences land on their balance sheet or result in their dismissal.
Open-weight models complicate this picture. Once weights are published, the vendor's control ends, and accountability shifts more heavily to whoever deploys and operates the model. That’s not a reason to abandon the framework: it’s a reason to make deployer accountability more explicit.
What aviation and automotive teach us
Aviation and automotive safety did not come from goodwill. It came from a combination of certification, investigation and liability that made failures expensive for the companies and leaders responsible. When Boeing's 737 MAX failed, the consequences reached the company, its leadership and its shareholders. That exposure can turn safety from a catch-phrase into an actual engineering requirement.Software has largely avoided this accountability. Vendors can disclaim liability through "AS-IS" license terms, and U.S. courts have historically treated software as a service rather than a product. That arrangement was tolerable when a bug meant a crashed application. It’s no longer tolerable when software can act autonomously inside financial systems and critical infrastructure.This has started to change. The EU's revised Product Liability Directive explicitly covers software, including AI. The 2023 US National Cybersecurity Strategy called for shifting liability toward software makers. U.S. courts have begun allowing product liability claims against AI companies to proceed. While we’ve charted a direction, it's not clear if the industry can prepare for it, or ultimately gets forced to change.Five steps toward meaningful AI accountability
- End the blanket "AS-IS" shield for commercial AI: Both vendors selling AI systems and end-users deploying them, especially agents that take actions, should not have the ability to disclaim all responsibility for foreseeable harm. We should not let contract terms override a basic duty of care.
- Apply a negligence standard, not a perfection standard: No model can be made jailbreak-proof, and a rule demanding that would stop anything from shipping. However, vendors should answer for known and foreseeable failure modes they chose not to address. Automotive law already works this way. Car manufacturers are not liable for every misuse, but they are liable for predictable ones they ignored.
- Hold deployers accountable, too: Responsibility does not end at the vendor. An organization that grants an AI agent unrestricted write access to production databases, with no approval gates or monitoring, has made a business decision. Its executives should own the consequences, just as they already answer for other material cyber risks.
- Let insurance companies price the risk: Insurers are good at one task regulators are not: attaching a price to risky behavior. Cyber insurers pushed multi-factor authentication into the mainstream by making it a condition of coverage. The same pressure can make controls such as scoped permissions, human approval for high-impact actions and monitoring a cost of doing business for AI vendors and deployers.
- Make accountability provable: Liability only works if causation can be established. AI systems that take actions should produce tamper-evident logs of what they did and under whose instructions. Operators should also have reliable ways to revoke an agent's access and halt it. Without this evidence trail, every incident becomes an argument about who did what.
