HalluSquatting: New AI attack method enables scalable botnets and large-scale infections

Based on information from Ars Technica, researchers have devised a novel attack method called HalluSquatting that exploits the inherent tendencies of large language models (LLMs) to hallucinate resource identifiers, potentially enabling the creation of massive botnets and large-scale device infections. This new threat targets AI coding assistants and agents, posing a risk to the growing ecosystem of AI-powered development tools.

HalluSquatting, short for adversarial hallucination squatting, leverages the inability of LLMs to accurately distinguish between legitimate and fabricated resource locations, according to a paper by researchers from Tel Aviv University, Technion and Intuit. Coding agents, which routinely pull code from external repositories, are susceptible because they can be tricked into accessing malicious resources that mimic popular ones. Attackers register these predicted identifiers and seed them with malicious code, such as reverse shells. This pull-based attack bypasses the limitations of previous prompt injection methods, which were typically push-based and limited in scale.

The attack is effective against numerous AI coding assistants, including GitHub Copilot and Gemini CLI. The researchers highlight that LLMs exhibit a high hallucination rate for newer resources, making them predictable targets. Once an attacker registers a squatted identifier, the LLM agent, when prompted to access it, will inadvertently download and execute the malicious payload. This could lead to the formation of large botnets for DDoS attacks or cryptocurrency mining, and enable scalable ransomware campaigns.

Source: Ars Technica