New AI security approach analyzes model internals

As reported by Dark Reading, researchers are developing a novel approach to AI security that moves beyond analyzing only the inputs and outputs of large language models (LLMs). This new method aims to understand the internal workings of AI systems by examining their activation patterns, offering a more robust defense against malicious use.

Offensive-security researchers are presenting a model-agnostic method for activation analysis at Black Hat USA 2026. This technique uses standardized rules for processing activation events, moving away from broad labels like "cybercrime" to a more granular scheme of cognitive elements (CEs). These CEs, such as "create content" or "personal information," can be combined into logical statements to detect specific threats like phishing attacks. The goal is to create an open system of identified CEs and rules, similar to Snort or YARA rulesets, to detect safety events.

This approach, dubbed GAVEL (Governance via Activation-based Verification and Extensible Logic), aims to identify specific activation patterns associated with granular objects and predicates. Unlike token-based analysis, activation analysis is language-independent and can detect malicious intent even when prompts are altered to evade content filters. While still a research project, this method offers a potential new layer of defense for AI systems, complementing existing security measures.

Source: Dark Reading