View More ASW Episodes

Prompting for Patches That Fix Vulns Without Adding New Ones – Keith Hoodlet – ASW #394

There's already an increase in volume of security flaws found by LLMs. And orgs are already turning to LLMs to write code. So, what happens when orgs lean on LLMs to create patches for those security flaws? Keith Hoodlet gives an exclusive early look at his team's recent research into the success, quality, and failures of LLM-generated security patches. Notably, they saw scenarios across a spectrum from robust, effective patches to patches that changed the software's behavior to patches that introduced new vulns to patches that didn't even fix the original vuln while also introducing a new vuln. The research considers factors like quality and correctness of prompts, complexity of the target ...

Full Show Notes
Segment One

Prompting for Patches That Fix Vulns Without Adding New Ones – Keith Hoodlet – ASW #394

There's already an increase in volume of security flaws found by LLMs. And orgs are already turning to LLMs to write code. So, what happens when orgs lean on LLMs to create patches for those security flaws? Keith Hoodlet gives an exclusive early look at his team's recent research into the success, quality, and failures of LLM-generated security patches. Notably, they saw scenarios across a spectrum from robust, effective patches to patches that changed the software's behavior to patches that introduced new vulns to patches that didn't even fix the original vuln while also introducing a new vuln.

The research considers factors like quality and correctness of prompts, complexity of the target software, programming language, and expertise required to understand what a robust patch should look like. If you're going to spend tokens on fixing security flaws, you want a feedback loop that fixes them correctly -- not an infinite loop of new flaws creeping in with every LLM iteration.

Watch for this research, its toolset, and data to be released on Thursday August 6th during Black Hat.

Guest
Application Security Manager at Thermo Fisher Scientific

Keith Hoodlet is the Application Security Manager at Thermo Fisher Scientific. He is the Co-Founder of the InfoSec Mentors Project .

Announcements

  • Threat intelligence should help you decide what to fix, but most of the time it’s disconnected from your code, your pipelines, and your actual risk.

    So how do you make it relevant to AppSec?

    At the Threat Intelligence Virtual Cybersecurity Summit on August 26th, learn how to apply intel to vulnerability prioritization and focus on what’s truly exploitable.

    Security Weekly listeners can register for free at https://securityweekly.com/threatintel using the promo code: CSS26-SW

  • InfoSec World brings cybersecurity professionals together across industries, from healthcare and financial services to government and the Fortune 500. Join the community in Orlando, October 12–14, for practical education, new perspectives, and cybersecurity research unveiled live. Listeners save 30% on their pass with code ISW26-SWSAVINGS at securityweekly.com/infosecworld2026.

List of Articles

Mike Shema

  1. Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident

    Kudos to HuggingFace for the transparency in sharing this timeline and details. Yes, there's an element of industry self-interest in terms of showing what advanced models can do, but it's also clear that there were misconfigurations and mistakes to which the LLM aspect was irrelevant.

    There's still plenty to evaluate and discuss in terms of LLMs, threat models, and how appsec may need to change. But the article's conclusion reflects how "…many parts of cybersecurity defense remain the same...strict isolation around evaluations, narrow trust boundaries, short-lived credentials, blocked metadata access, and detection capable of quickly correlating activity across systems to keep up."

  2. OpenAI’s Rogue AI Agent Hacked More Than Just Hugging Face | WIRED

    There's a lot of overlap among the media coverage of the OpenAI hack of HuggingFace. It's helpful to see which aspects of the timelines get the most attention and how different articles frame them.

    For completeness, here are some additional articles that appeared this week:

    And not to be left out of the attention that OpenAI got with this, Anthropic revealed a similar situation where its own models attacked third parties — https://therecord.media/anthropic-ai-hacked-three-real-companies

  3. FastJson 1.2.83 Remote Code Execution

    I like this article because it represents another example of an engaging writeup of a technical flaw. Plus, finding and exploiting the flaw was quite a journey of stubbornness and insights.

    It also made me wonder what the primary factors are in preventing orgs from making an update to FastJson 2.0.x given that it's been available for about four years. And that can serve as a thought exercise over a spectrum of inertia, unmanaged and orphaned software, upgrade-blocking bugs, priorities, or even something else I can't think of.

  4. How we use /goal to find bugs in Patch the Planet

    This felt like a good complement to our interview segment. It's about finding bugs, which would apply to double-checking LLM-generated patches and perhaps even creating the patches in the first place.

  5. Scrutineer: Scanning Open Source Without Flooding Maintainers

    Everyone is releasing their approach to finding software flaws with LLMs and agents. Here's how the OpenSSF is doing it.

  6. Introducing MAI-Cyber-1-Flash inside MDASH

    I only included this because of its subtitle, "World-class security at half the cost"

    It's nice to see a growing attention towards cost related to LLMs. Now on to fitness for purpose and quality.

Show More

Stay in the Know, No Smoke and Mirrors – Join Our Newsletter

Get expert insights and technical breakdowns straight to your inbox.
Join Now

Related Episodes

Related Content

You can skip this ad in 5 seconds