The recent popularity of MCPs is surpassed only by the recent examples deficiencies of their secure design. The most obvious challenge is how MCPs, and many more general LLM use cases, have erased two decades of security principles behind separating code and data. We take a look at how developers are using LLMs to generate code and continue our search for where LLMs are providing value to appsec. We also consider what indicators we'd look for as signs of success. For example, are LLMs driving useful commits to overburdened open source developers? Are LLMs climbing the ranks of bug bounty platforms?
In the news, more examples of prompt injection techniques against LLM features in GitLab and GitHub, the value (and tradeoffs) in rewriting code, secure design lessons from a history of iOS exploitation, checking for all the ways to root, and NIST's approach to (maybe) measuring likely exploited vulns.
Mike Shema
- Remote Prompt Injection in GitLab Duo Leads to Source Code Theft
You know when an article starts with a question like, "Could we embed hidden instructions in different parts of a GitLab project and still influence Duo’s behavior?" that the answer is going to be, "Yes."
But it's fun to see the process of getting from a threat modeling question to a demonstrated exploit. This also falls into the ever-growing category of attacks that show the erasure of code vs. data boundaries and how LLM prompts come from anywhere and everywhere.
- GitHub MCP Exploited: Accessing private repositories via MCP
If the 2010s were marked by new names for JavaScript frameworks every week, the 2020s will be remembered for new names for LLM attack techniques every week. Here we have an article that coins (and does a great job of explaining!) a technique they call "Toxic Flows".
The attack not only demonstrates more of the prompt injection problem of code and data being mixed, but also the challenge in reasoning about and enforcing separation of privileges for agents.
- The AI Hangover Era (The Everything App Part 3)
This article is more about AI, LLM adoption, and hype than it is about anything related to appsec. But I grabbed it because it speaks to a search for value from LLM code generation (or other uses), which is something I've been looking for in terms of LLMs as appsec assistants.
There's also only so much that I think we can repeat week after week about yet another prompt injection attack or another example where LLM agents have commingled data and code in a way that cannot be fixed by some sort of LLM equivalent to SQL's prepared statements. We'll cover AI because there's a lot of consequences for appsec from AI adoption. But there's also so much...emptiness once you get beyond the novelty of seeing autocomplete for code scaffolding or common patterns.
- How Swift’s server support powers Things Cloud
I love talking about refactoring, rewriting, and otherwise revisiting code. Understanding how and when teams chose to switch frameworks, designs, and even languages reveals a lot about developer priorities and decisions. It also sheds light on how to influence decisions and where security falls into design considerations. In this example, a team chose to rewrite a service from Python and C into Swift. The primary benefits were reduced latency, reduced compute costs, and simplified maintenance. Oh, and it just so happened to add better memory safety along the way.
- Tachy0n
Don't worry if the majority of this article gets into too much detail or background on iOS exploits. Focus on parts 4 and 5, the "Aftermath" and "Conclusion" sections. They explain how the past five years of iOS design has shifted from fixing a single bug report to defending against the technique that bug relied on. It's a great illustration of the modern appsec emphasis on killing off classes of vulns with secure design. And it's very nice to see practical examples of that approach.
- Root in prod: The most important security analysis you will never do on your AWS accounts
Complexity, privileged access, and over-provisioned accounts existed in the cloud before LLM agents ever came on the scene!
- CSWP 41, Likely Exploited Vulnerabilities: A Proposed Metric for Vulnerability Exploitation Probability | CSRC
I will be a fan of anything that helps orgs reduce a list of everything with a CVE to a list of patches that are worth spending unplanned time on.