Browser extensions could be used to exploit AI tools, researchers say

Malicious browser extensions could inject prompts to AI tools and extract potentially sensitive data, LayerX security reported Tuesday.

AI tools that are accessible from a user’s browser could be manipulated by extensions that are able to read from and write to a page’s Document Object Model (DOM). The prompt input field for large language models (LLMs) is typically part of the DOM of a webpage, according to LayerX.

Researchers demonstrated that this exploit, which LayerX called “man in the prompt,” can be achieved by extensions with no special permissions against popular AI tools like OpenAI’s ChatGPT and Google’s Gemini.

In one demonstration, the researchers used an extension to open a ChatGPT tab in the background, send prompts asking for sensitive information to the chatbot from a command-and-control (C2) server, and exfiltrate the chatbot’s responses to an external log. They were also able to delete the chat from the user’s account, removing evidence of the interference.

In the case of Google Gemini, a malicious extension could take advantage of Gemini’s access to other Google Workspace resources to gain information from Gmail emails, Google Drive documents, user contacts and more. LayerX said it reported its findings regarding the Gemini exploit to Google.

Furthermore, internal AI systems that are accessible from a web browser can also be subjected to the same exploit, giving attackers access to sensitive details such as intellectual property, internal communications, contracts, financial documents and more, LayerX reported.

LayerX research showed that about 99% of enterprise users have at least one browser extension installed, and more than half have more than 10 extensions installed. Malicious or compromised extensions can potentially fly under the radar of enterprise security systems, especially when they do not request any special permissions from the user.

The company said businesses can combat this type of exploit by monitoring extensions’ DOM-level interactions with GenAI tools and looking for listeners or webhooks that could interact with AI prompts. Additionally, LayerX said extensions should be blocked based on behavioral risk rather than static assessments.

“Since a static assessment based on permissions will not suffice (since some extensions won’t require any permissions), combining publisher reputation with dynamic extension sandboxing is the best way to detect risky and malicious extensions,” LayerX said.

SC Media reached out to LayerX for more information about its “man in the prompt” exploit and did not receive a response. SC Media also reached out to OpenAI and Google for their comments on LayerX’s research and did not receive a response from either company.