Microsoft warns of AI recommendation poisoning attacks

Microsoft has issued a warning to customers about a new technique called AI Recommendation Poisoning, which manipulates AI technology to produce biased advice. Security researchers have detected a surge in attacks designed to inject manipulative data into the memory of AI models, a method that can subtly influence user recommendations on critical topics. This vulnerability poses a significant risk to the integrity of AI-driven insights, as reported by The Register.

The attack involves embedding hidden instructions within URLs that link to AI chatbots. These instructions, often included as query parameters, can alter the AI's output. For instance, a prompt could instruct an AI to summarize an article in a specific style or with a particular bias. Microsoft identified over 50 unique prompts from 31 companies across 14 industries, noting that freely available tools make this technique easy to deploy. The manipulation can be persistent, as poisoned AI models may treat injected instructions as historical context for future responses, influencing subsequent interactions.

Microsoft advises users to be cautious with AI-related links, review AI memory, and question dubious recommendations. For organizations, scanning communication channels for poisoning attempts is also recommended.

Source: The Register