As it was a year ago, AI was the dominant theme of the OWASP Global AppSec USA conference last week in San Francisco. That's not surprising because the Open Worldwide Application Security Project has put itself at the forefront of developing guidelines for secure AI development.
There wasn't any unveiling of a new OWASP Top 10 vulnerabilities list, because those can take years to compile. But the kickoff keynote address by Root Evidence's Jeremiah Grossman covered a topic potentially even more momentous: a likely "cybersecurity reckoning."
Grossman explained that the day will soon come when cyber-insurance providers will have analyzed enough data about breaches, intrusions and other incidents to know which security practices, products and vendors add value to their clients — and which are mostly useless.
"Our industry works on compliance and FUD," Grossman admitted. "How many of our security dollars are wasted every day chasing theoretical super-hacker risks that don't result in any breach ever?"
As the old saying goes, when the tide goes out, you find out who's been swimming without a bathing suit. Grossman said cybersecurity vendors that want to survive will have to show tangible results from use of their products and should be ready to offer monetary guarantees if their products fail to defend customers.
He wasn't that worried about AI, however. As he sees it, AI is "changing volume, not velocity" regarding the number of threats, and the old means of defense still work as long as they can keep up.
Resilience as a rule
Another significant change will come because of the European Union's Cyber Resilience Act (CRA), warned Adam Shostack, who created Microsoft's threat-modeling framework and now leads his own consulting firm.
"The cloud was transformational in how quickly we moved, how we deployed, how we rolled things out," said Shostack. "I believe CRA will be that impactful."
The CRA, which goes into full effect in December 2027, applies to most devices with an internet-connected digital component sold in the European Union: smartphones, computers, operating systems, and "smart" appliances like TVs, refrigerators and baby monitors.
"The coverage is extremely broad," said Shostack. "It covers nearly every item that connects to the internet."
Many of the CRA's mandates are common sense, he pointed out. Customers must receive clear instructions on how to disable automatic updates, perform manual updates, restore factory settings, and contact manufacturers.
It does create more work for manufacturers and vendors. Risk assessments are mandatory, even for products already on the market. Each software update will be subject to an individual assessment. And manufacturers must archive substantial documentation on each product for up to 10 years after the product leaves the market.
But the CRA can be quite confusing. Purely digital services that don't require software installation on a physical device, such as software-as-a-service (SaaS) applications, are not covered. But if your SaaS service provides a smartphone app as an interface, then the CRA does apply.
Motor vehicles, medical devices, and civilian aircraft are excluded because other regulations already cover them. Free and open-source software, like many distributions of Linux and related applications, are in theory also exempt — but any paid services providing support for open-source software are not.
Or then again, maybe open source doesn't get a pass.
"Despite 15 pages of guidance, it's not clear," said Shostack.
What is clear is that the CRA provides a tremendous opportunity to advance application security, which is OWASP's main goal.
"Almost every OWASP project is a CRA project now," he said, adding that OWASP members "should be serving those who are affected by the CRA."
How safe can agentic coding get?
Several presentations delved into the potential dangers of using AI to write code. Dan Lisichkin of Pillar Security cautioned against giving AI agents too much leeway in development environments but conceded that they are inevitable.
"Agents are proven to have security gaps that can lead to exploits," Lisichkin said. "But agents are not going away. We need to learn to integrate them securely into our products."
He pointed out that blocklisting and allowlisting (or blacklisting and whitelisting) resources to manage AI agent access doesn't really work. Agents can get around the former, and the latter doesn't control what agents can do with permitted resources.
"To build safer sandboxes, use containers and VMs whenever possible," he said.
And, he added: "Every tool is its own attack surface. A tool should do just the narrow job it exists for."
In another talk, Shruti Datta Gupta and Joseph Seasly of Adobe described their efforts to make AI coding agents produce more secure code.
They designed an in-house harness to "steer" the agents toward better practices, ran the results through static application security test (SAST) analysis, and compared those to the code generated by "unsteered" coding agents.
They found that various forms of steering reduced the number of unsafe coding errors by roughly 40%, with some variation, but little more than that.
While there was more positive impact on spotting what Gupta and Seasly called "textbook bugs," the rate of other common flaws, like hardcoded secrets and server-side request forgery (SSRF) "barely moved."
Using steering harnesses is nonetheless worthwhile, they added, because even a 40% reduction in software flaws is valuable. But they advised building the harnesses internally and slowly, testing each new step, and re-establishing baselines with each new LLM model release.
"Don't trust the model, don't trust what the frontier labs say," said Seasly. "Trust your own measurements."
Shifting left in a circle
Finally, Manoj Kumar Yuvanesh and Uday Bhaskar Seelamantula of Autodesk discussed how automated code generation is creating an unexpected bottleneck and proposed a new model for security reviews.
"Code is being generated at a pace that humans can no longer keep up with," said Yuvanesh.
In traditional application security, he explained, developing software is expensive, but verification cost less than implementation, and shifting security "left" — as close to the beginning of a project as possible — overall reduces cost.
Security tests are performed at the end of each stage, requiring development teams to pause, review and approve changes before proceeding to the next stage.
But with AI coding, Yuvanesh said, the volume of code being generated is drastically higher — and the cost of development is drastically going down. The human factor is greatly reduced.
"AI native software development behaves like a software factory," he said. "The system can create, judge, and review its own code before a human ever sees it. The human may see only the final pull request."
Yet that final human security review is necessary — and it's creating traffic jams.
"AI can generate a fix in seconds, but the review queue still moves at human speed," said Yuvanesh.
AI agents need to be brought in to perform more of the review process, explained Seelamantula. But Yuvanesh said the reviewing agent needs to be completely independent of the coding agent, and security reviews need to become continuous, not intermittent.
"In human-paced software development, the process is linear and there's a left to move to," said Seelamantula. "AI-paced software development is a circular loop, so the security has to be enforced on every pass of the loop."
He emphasized that humans still cannot completely trust AI agents and must use more than one AI agent to verify fixes.
"Trust the evidence over the confidence," Seelamantula said. "Confidence is what the model believes; evidence is what it did."
