Google reports first known AI-assisted zero-day exploit in the wild

The Google Threat Intelligence Group (GTIG) reported the first known case of a zero-day exploit being created using AI by threat actors in the wild.

The company said Monday that a group of “prominent cyber crime threat actors” used a large language model (LLM) to create a Python script that exploits a two-factor authentication (2FA) flaw in a “popular open-source, web-based system administration tool.”

The exploit was determined with “high confidence” to be LLM-generated or assisted, based on the presence of a hallucinated CVSS score and several of educational docstrings, as well as a structured Python format known to be characteristic of LLM training data, GTIG said.

Google did not name the threat actors suspected to be behind this exploit, nor the affected open-source project, but noted that it did not believe Gemini was used in the exploit’s creation. GTIG reported the vulnerability and exploit to the affected vendor and worked with the vendor to disrupt the malicious activity.

“This signals a shift from human-paced vulnerability discovery to machine-scaled weaponization, a transition security leaders have long anticipated but failed to operationally absorb,” Ronald Lewis, head of cybersecurity governance at Black Duck, told SC Media.

The revelation comes after the announcement of Claude Mythos Preview escalated fears that attackers could soon use advanced LLMs to discover and exploit new vulnerabilities much faster than they could be patched. Anthropic released the model preview only to select companies through a program called Project Glasswing due to concerns that its ability to autonomously develop zero-day exploits could be misused.

It is unclear when the exploit GTIG described was created, but Acalvio CEO Ram Varadarajan told SC Media the report demonstrated that “AI-powered cyberattacks have moved from theory to reality.”

“Modern models no longer just scan code for technical mistakes. They can infer what developers intended the software to do and spot contradictions humans missed. That makes a new category of vulnerabilities far easier to find: hidden business-logic flaws, broken trust assumptions, and authorization errors that appear perfectly valid to conventional security tools but can still be exploited,” Varadarajan said.

Just last year, GTIG released a report on the use of Google’s Gemini AI model by state-sponsored threat actors that concluded attackers’ use of LLMs was mostly experimental and limited to “productivity gains” rather than “novel capabilities.”

In the most recent report, Google described an increasing interest in AI-driven vulnerability discovery and exploit development, particularly by threat actors sponsored by China and North Korea, as well as the increasing use of AI in attack orchestration and the development of evasive malware.

The report highlighted two previously discovered and two newly disclosed malware families that leverage AI for evasive techniques such as polymorphism and the use of benign decoy code. The previously described PROMPTFLUX and HONESTCUE families leverage the Gemini API to dynamically generate and modify malware code, while the newer CANFAIL and LONGSTREAM families — used by Russia-nexus threat actors targeting Ukrainian organizations — leverage LLMs to generate large volumes of decoy logic that obfuscate the malware’s malicious intent.

GTIG also highlighted PROMPTSPY, a novel Android malware family first discovered by ESET, that abuses the Gemini API and accessibility features to interact with the Android user interface (UI) in an automated fashion. The use of an automated Gemini agent enables the malware to manipulate the UI in real-time in response to user activity.

“Today, this type of AI-enabled malware is noisy and consequently easy to see. As attackers’ capabilities with AI continue to advance, those attacks will become easier to mask,” noted Nicole Carignan, SVP of security & AI strategy and field CISO at Darktrace, in an email to SC Media. “Defenders need to adapt away from security approaches that expect attacks to contain set signatures, and towards one out of place behavior.”

Along with these novel, emerging applications, attackers continue to use AI for research, reconnaissance, and the generation and translation of content for social engineering and disinformation campaigns. The researchers noted that attackers are increasingly moving tasks such as victim reconnaissance beyond simple chatbot interactions to agentic workflows that scale and accelerate their malicious operations.

Attackers also increasingly abuse tools such as custom middleware, proxy relays and automated registration pipelines to gain anonymous and consistent access to premium-tier LLM models, including by abusing free trials and cycling through disposable accounts.

“There are things that cyber defenders can do to improve their defenses against AI-driven threats. Security teams must deploy platforms capable of safely automating the remediation process, such as pushing verified firmware updates to thousands of OT endpoints simultaneously,” John Gallagher, VP of Viakoo Labs, told SC Media.

“While attacks may be fully autonomous, defense should rely on AI-enabled precision and speed for human decision-makers,” Gallagher continued. “AI should serve up the remediation options, with human operators making the critical approval decisions.”