As much as I hate everything about the rise of LLMs, saying this isn't impressive because it can be matched by "an elite security researcher" isn't very reassuring to me. It's still an agent being pointed at a codebase and finding hundreds of vulnerabilities. Even if only a twentieth turn out to be exploitable in practice, that's still a terrifying tool to imagine in the hands of hackers who might otherwise lack the skills to find these vulnerabilities.

Most hacking groups buy exploits off of dark markets and indiscriminately target servers until they find one that's vulnerable. The number that can actually develop those hacks is far smaller, but if you can simply ask an LLM to find a vulnerability then that bar is lifted. Hell, you could probably coerce it into writing the actual exploit too by claiming you need a proof-of-concept for a CVE writeup.