President Donald Trump pushed back Sept. 14 on Anthropic CEO Dario Amodei’s call over the weekend for a slowdown in frontier AI development.While AI leaders such as Open AI’s Sam Altman and Elon Musk of SpaceX agreed with Amodei, Trump said there's no reason for federal oversight and that the U.S. is in an AI race against China, adding there is a "sick conspiracy against AI and data centers."In making his case on Truth Social, Trump stayed true to form: “The only control or 'guardrails' that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that, in spades!”Security pros were predictably skeptical of any slowdown, pointing out that any such pause would require authoritarian countries such as Russia and China to comply. Some also called Amodei’s notion that AI agents could take over the internet in six to 12 months as mostly hyperbole.“Taking over the entire internet is a huge leap,” said Diana Kelley, chief information security officer at Noma Security. “The credible near-term warning is that poorly controlled agentic systems could operate large botnets and cause substantial damage. That’s why it’s critical that we govern and secure agentic systems now, not only through behavioral guardrails, but through deterministic controls such as least privilege, network segmentation, tightly restricted internet access, and air gaps where appropriate.Dana Simberkoff, chief risk, privacy, and information security officer at AvePoint, said while we've all heard AI safety warnings before, especially from Elon Musk, this one’s different — and it's serious.“It's hard to overstate how serious this is,” said Simberkoff. “We don't fully understand why these systems behave the way they do, and the industry is advancing faster than our ability to understand or control what it's building. That's not a car-without-a-seatbelt problem anymore, it's a call for real regulation, independent oversight, and enforceable guardrails before, not after, something goes wrong. Even a scenario short of worst-case could be catastrophic. The fact that industry leaders are the ones sounding this alarm, and asking to be held to it, is a positive sign. But it only means something if the controls follow."
“The balancing with any legislation is going to be finding the sweet spot between ensuring the speed of innovation doesn’t go away while ensuring guardrails are in place,” said Gerry. “Collaboration between the private and public sector is critical here.”
Frontier AI models trained on known research
On the claim that AI could take over the internet within a year, Derek Banks, senior security researcher at Black Hills Information Security, said that AI agents up to this point have not been observed creating new or novel attack techniques or classes beyond what human attackers can do and have already done.“That makes sense when you consider how these systems are built,” said Banks. “Frontier models are trained on human generated cybersecurity research, exploit code, and programming data. They reflect what the security community has already published. They are not a source of some alien capability.”Banks said what researchers have seen change with modern LLMs is the speed and scale of attacks: so if an AI agent could take over the internet six to 12 months from now, human attackers could exploit it right now, just not at the scale or speed of AI agents.“The takeover scenario is not a new capability appearing out of nowhere,” said Banks. “It’s the attack surface we already have, getting hit faster and more broadly than defenders are prepared for.”Banks also pushed back on the alleged scenario where agents spin up their own message boards and agendas. He said it makes for a good headline, but it’s not what his teams see on real engagements.“We see attackers using AI to do known human techniques faster and at larger scale,” said Banks. “The defense against that is not exotic. It’s the same fundamentals the industry has under invested in for years, now with a much shorter grace period for getting them wrong.”Christopher Jess, senior R&D manager at Black, added that the industry should take all warnings seriously without presenting the worst-case outcome as an established fact.Jess said AI does not need to become superintelligent to create serious security problems. The immediate threat comes from AI assisting individuals and organizations in creating and running attacks.Black Duck’s State of AI-Powered Software Development report found that 92% of surveyed teams report productivity improvements from AI coding assistants, yet only 30% have full governance in place. Organizations should close that gap by approving and monitoring AI tools, limiting agents’ access to sensitive systems, testing generated code and third-party components, and requiring human approval for high-impact actions. Faster software delivery must not mean less accountable software delivery.“The debate should not be reduced to unrestricted development versus stopping everything,” said Jess. "We should require independent testing of high-risk capabilities, timely incident sharing, and enforceable safeguards. Where risks cannot be adequately controlled, access or deployment should be restricted. We should make safety a condition of progress, not an optional extra in the race to lead AI.”Anthropic and OpenAI incidents a wake-up call
Dave Gerry, chief executive officer at Bugcrowd, said while he’s not worried about the machines waking up one morning and deciding to end humanity, he’s worried about a system with too much access doing exactly what it was told without the necessary adversarial testing.Gerry said we’ve already seen both Anthropic and OpenAI reveal that their agents have gone outside of the testing environments they were in. As a result, Gerry said OpenAI has called for action on the national level: if the designer of the model thinks outside oversight is necessary, that should be a wakeup call for all of us to act quickly, said Gerry,Here's how Gerry thinks the industry should respond:- Give limited access to the agents as necessary — true for both individual or personal use, and corporate use.
- Remain a part of the decision-making process and ensure that there’s human oversight into the critical decision or action points.
- Don’t fully trust these systems immediately. The same way that we don’t fully trust a new employee on day one and trust must be earned, the same is true for new software we’re deploying.
- Conduct independent adversarial testing prior to launch to ensure that the models, and resulting tooling, are reasonably safe to use.