AI backdoor threats: Detecting ‘sleeper agents’ in large language models

As detailed in The Register, a new and concerning security threat has emerged in the form of "sleeper agent" backdoors embedded within artificial intelligence large language models. These hidden vulnerabilities, inserted during the model's training phase, can be activated by attackers using specific trigger phrases, leading to potentially malicious outputs.

Attackers embed these backdoors into a model's weights, making them difficult to detect. Once activated by a predefined phrase, the model performs a malicious action. Researchers at Microsoft's AI red team have identified three key indicators of such poisoned models. These include a "double triangle" attention pattern where the model disproportionately focuses on the trigger phrase, a tendency for the model to leak its poisoned training data, and the "fuzzy" nature of the backdoors, meaning partial trigger phrases can still activate them.

The difficulty in detecting these sophisticated backdoors, likened to finding a "golden cup," highlights a continuing challenge for AI security.

Source: The Register