Bad actors exploit X’s Grok to run ‘racy’ malvertising campaign

Bad actors are exploiting X’s AI assistant Grok by embedding malicious uniform resource identifiers (URIs) within the "From" metadata of video-based advertisements, which lets them promote posts with mostly sketchy adult content baits.

Here's how it works: In many instances, these same bad actors prompt Grok to identify the video ad's source, which then parses and presents the hidden URI as a clickable link in its response. In doing so, it bypasses security protocols, thus amplifying the link's credibility and reach to millions of unsuspecting users.

“This tactical subversion of a trusted system component essentially transforms Grok into an unwitting accomplice for malvertising, yielding a significant return on investment for the perpetrators while evading conventional detection and mitigation measures,” said Nic Adams, co-founder and CEO at 0rcus.

Nati Tal, a threat researcher from Guardio Labs added his explanation in a Sept. 3 post on X: “A malicious link that X explicitly prohibits in ads (and should have blocked entirely!) suddenly appears in a post by the system-trusted Grok account, sitting under a viral promoted thread and spreading straight into millions of feeds and search results!”

Andrew Bolster, senior R&D manager at Black Duck, described this recent scenario involving Grok as the most recent demonstration of the Lethal Trifecta: an emerging term within the AI security world used to categorize high-risk AI targets if they combine three critical capabilities: access to private data, external communications, and exposure to untrusted content.

“Grok naturally operates in the overlap of these factors, and with its added social/algorithmic ‘weight’ – has become a natural target for manipulation and exploitation,” said Bolster. “From a security perspective, these types of attacks are more akin to social engineering than traditional security breaches. However, whether an intruder breaks into your office through the receptionist or through the window, you’ve still been breached.”