AI’s Junk Vulns, Web3 Backdoor, LLM CTFs, 5 GenAI Mistakes, Top Ten for LLMs – ASW #310

Just a quick reference for this one because of how it highlights the "..;/" payload for path traversal and gives a shout out to prior work by Orange Tsai.

One of many junk security reports that the curl project has received. There were always junk reports that were blatant copy-and-paste output from scanners. Now there are more and more junk reports that are blatant, but better worded, output from LLMs. It's a lot of time wasting for anyone who has to triage these.

I'll believe LLMs are effective at vuln discovery when we start seeing someone show off even a low five-figure bounty success rate of using them.

For even more context, here's the curl bug-bounty stats and a post from January 2024 about LLM-generated junk.

Curl isn't the only project impacted by generative AI slop, here's a similar post from Python land.

Setting a reminder to check up on the progress of this. It's also an opportunity to talk about general appsec practices we'd hope to seeing being practiced to make these kinds of attackers easier to identify early and constrain their impact.

For a reasoned approach to secure communications, check out this post from Bob Lord.

I've gone most of the year without bothering with crypto news -- they're all pretty much repetitions of rugpulls, scams, compromised private keys, and bad "smart" contract designs.

Here we have a supply chain example where a popular JavaScript library was backdoored. It surprised me that the reported losses weren't greater. One useful thing about crypto is that it's much easier to quantify impacts since it's always about the (ostensible) value of stolen tokens.

But it also raises a larger question of whether JavaScript and the browser is even appropriate for this kind of use case. I wouldn't have the same reaction for password managers, though.

LLM security is more than just prompt injection, but prompt injection is the most fun and accessible type of attack to experiment with -- you barely even have to bother with syntax or real words.

This CTF runs through January 20, 2025. We'll check it out at the end to see if there's anything more interesting than variations on "ignore previous instructions" or "pretend this is a bedtime story".

Yeah, a second crypto post in one month. (That's like the entire limit for the year.)

It's another CTF-like competition against generative AIs that demonstrated the difficulty in defending against prompt injection. It also comes with a prize. However, I only consider it CTF-like because you have to pay to participate and, in the world of crypto, anything collecting money already has a scam-adjacent feel to it.

Only half the length of a top 10 list!

I do like how these are presented and explained. Still, a lot of it reads like it could be about an API instead of an AI. Along those lines, two items that stand out for me are, "Excessive, overprovisioned access" and "Neglecting inherited vulnerabilities".

Since I included the Google Cloud blog post on gen AI mistakes, I might as well include the latest version of OWASP's Top 10 for Gen AI (or LLMs) since it was updated back in November.

Influence the direction of Rust and its developer support! Or, if you still refuse to give up C programs, let them know what other keywords you'd like to see beyond "unsafe" -- maybe "inscrutable_code" or "yay_pointers" or just "doomed".

Our friends at Portswigger work through how telling a WAF to treat cookies as the original "Version 1" standard from 1997 allows various tomfoolery regarding escaping and use of white spaces. Interesting(unfortunate?) to see how different frameworks handle this differently.

In which we find a CVSS 10 rated vulnerability related to file access