COMMENTARY: It’s been more than two years since Microsoft patched the zero-click EchoLeak vulnerability, but the root problem remains: EchoLeak was a flaw that showed how an attacker could use a crafted email to exploit AI’s capabilities.
While answering a routine question, Microsoft Copilot read this email, followed its instructions, and leaked sensitive data through an image the browser loaded on its own.
[SC Media Perspectives columns are written by a trusted community of SC Media cybersecurity subject matter experts. Read more Perspectives here.]
LLMs can’t distinguish between instructions and data. Copilot only wrote a response, and chaos ensued. We are at the tail end of 2026, moving into 2027, and agents today can send email, update records, call internal APIs, and do a whole lot more. The underlying weakness can result in a much bigger blast radius. Here are five lessons from EchoLeak that can help minimize this radius:
1. A model refusing an instruction is not a sufficient security control.
All models are vulnerable to prompt injection, and vendors acknowledge it. OpenAI disclosed that its own models could be tricked by injections that copy themselves into the agent’s output.
Models have come a long way since EchoLeak, yet they still cannot stop these attacks on their own. Defenders have their job cut out for them. They must thwart every attempt, whereas attackers need to succeed only once. Even a failed attempt teaches them something, raising the odds that the next attack will succeed. A model's built-in defenses are not enough to repel such attacks, so they must get reinforced with additional guardrails that complement the ones already in place.
It's a built in problem. The model represents both the guard and the target. Attacker instructions arrive mixed in with legitimate content. The model’s own judgment, which can vary from one attempt to the next, decides whether to follow them. Plan for the possibility that an attack will bypass built-in protections by way of introducing additional guardrails independent of the model itself.
2. Don’t build a defense solely on known attack patterns.
Guardrails often use AI classifiers trained to recognize prompt-injection patterns, including phrases such as “ignore previous instructions” or “IMPORTANT SYSTEM UPDATE.” These classifiers are trained on such phrases and can only identify patterns that existed during its training. Attackers can always find phrasings that are out-of-distribution of the training set for those models, which will allow them to bypass this trained classifiers approach.
EchoLeak bypassed a dedicated cross-prompt injection classifier without mentioning AI, Copilot, or instructions. Instead, the attacker wrote the email as a request to its human recipient. To the classifier, it was a normal business email; the model interpreted it as a task to carry out.
Think of attack-pattern recognition as analogous to patching known vulnerabilities. Both are necessary, but not sufficient. Attackers can create new variants faster than defenders can catalog them, and most are never disclosed. Teams then to make pattern recognition just one of the defensive layers – and not the foundation of the defense.
3. Defense still revolves around least privilege.
By limiting the tools an agent can call, the data it can reach, and its features, teams can shrink the attack surface. In EchoLeak, the data left through an image. Copilot did filter links and images, but the attackers slipped past these checks. The irony here: most users never needed Copilot to show images in the first place. With that feature off, the attack would have had no way to get the data out.
For every capability an agent has available to it, ask:
- Does it need this capability to do its job?
- Does it need it all the time, or only in specific situations?
- How detrimental is it to disable it by default or require user approval?
4. Enforce rules in code, not in prompts.
LLMs are powerful, but relatively easy to manipulate. If there’a clear-cut rule, enforce it in code, not in the prompt.
Suppose we deploy an HR agent to answer compensation questions. A common mistake is allowing the agent to access all employee records while relying only on a prompt telling it to show each person their own data. A malicious instruction may cause the model to ignore that prompt-level restriction.
Instead, enforce the rule in the application. Check in the application’s code who has signed-in and make sure the application can only ever impersonate the logged-in user, and does not get granted broad data access. The model should never decide whose data it’s authorized to retrieve.
5. Use layers of protection.
Keep layering protections until exploitation stops paying off. Let’s take a trip down memory lane. Back in 2000, attackers would often exploit a buffer overflow, a bug that let them overwrite a program’s memory and take control of it. It used to take minutes to exploit such a flaw. The bugs did not go away, but operating systems kept adding defenses. To name a few: stack canaries detect when memory has been overwritten; DEP stops injected data from running as code; ASLR shuffles memory addresses so attackers can’t predict where to aim. Together, these defenses blocked many common attack paths and made exploitation exponentially harder.
AI agents need the same approach. Layer safeguards so that finding one weakness does not give an attacker everything they need. These safeguards can include tracking where content came from, controlling where outputs can go, and checking tool calls against policy before they run. Require human approval for high impact actions and monitor for behavior that does not match the user’s task.
Microsoft fixed EchoLeak, but its lesson still applies. Prompt injection isn’t going away anytime soon. Teams need to layer different types of protection: attack detection, least-privilege access, code-enforced policies, output controls, and human approval, so that a bypass of one defense does not become a compromise of the whole system.
Itay Ravia, Head of AI Labs at Cato Networks
SC Media Perspectives columns are written by a trusted community of SC Media cybersecurity subject matter experts. Each contribution has a goal of bringing a unique voice to important cybersecurity topics. Content strives to be of the highest quality, objective and non-commercial.
