New attack bypasses AI guardrails by encrypting malicious prompts

A new data theft hack has been devised against Elon Musk's Grok AI, employing a deceptively simple trick to force the large language model to steal user chats and other personal information. This follows a similar attack against Microsoft 365 Copilot earlier this week, where a secret input caused the AI assistant to exfiltrate a password. Despite being informed of the Grok vulnerability in June, the assistant continued to leak data at the time of reporting, according to a recent report by Ars Technica.

The attack, developed by Rony Utevsky, a researcher at Adversa, is dubbed "cryptographic context injection" and exploits a fundamental weakness in how AI models process information. Attackers encrypt malicious instructions, embedding them within content that the AI is asked to summarize or process. Because AI models like Grok are designed to comply with user requests, they can be tricked into decrypting and executing these hidden commands. The encrypted instructions, along with a decryption key, are provided on a webpage. When the AI attempts to summarize this page, it decrypts the instructions, which then direct it to construct a fake decryption key. This fake key is actually the user's personal data, which is then sent to an attacker-controlled URL.

Researchers believe that current AI safety guardrails, which typically scan input as static text, fail to detect these encrypted commands because they do not execute code or decrypt content during the initial inspection. This method has also been used to bypass safety rules in Google's Gemini, demonstrating a broader trend of attacks targeting the wider context AI models treat as their own, beyond simple prompts.

Source: Ars Technica