Myths and Mythos: An AI Vulnerability Research FAQ

Last week, Anthropic announced two rather large things: a new frontier model called Claude Mythos that it said is very good at finding new vulnerabilities and writing exploits; and Project Glasswing, a large-scale coordinated vulnerability disclosure effort to publish the bugs that Claude Mythos has discovered. These announcements set off a classic infosec cycle of concern, backlash, backlash to the backlash, etc. It’s still very early days for all of this, and relatively little is known about Claude Mythos’s real capabilities, so we wanted to dig into what actually is known and what it all means. 

What exactly is Claude Mythos?

Claude Mythos is a frontier model from Anthropic, a company you may know from such hits as Claude, Claude Code, and its nasty public dispute with the Pentagon. Claude Mythos, as the label “frontier model” might imply, is on the leading edge of LLM design and performance and Anthropic has said that it is not planning to open the model up to public use because it is too advanced (more on that later). Think of Claude Mythos as an experimental high-end plane like the SR-71 Blackbird that does really sick stuff that’s none of your business. 

So how is that different from other advanced LLMs doing cutting-edge things?

We’re not sure! Like other privately developed models, Claude Mythos is opaque and its inner workings and training sets aren’t available for public scrutiny. What we do know is that Claude Mythos is really good at finding vulnerabilities. We know that because the company turned it loose on a large number of products from vendors such as Apple, Cisco, CrowdStrike, Google, Microsoft and others (with the cooperation of those companies) and told it to go find new vulnerabilities and write exploits for them. Which it did.

“Claude Mythos Preview demonstrates a leap in these cyber skills—the vulnerabilities it has spotted have in some cases survived decades of human review and millions of automated security tests, and the exploits it develops are increasingly sophisticated,” Anthropic says.

So that’s cool. But security researchers already have had success doing this kind of work with other LLMs, including Anthropic’s own Opus and Claude models. The difference is that Claude Mythos seems to be better and more efficient at finding bugs, especially deep-seated ones.

How does Project Glasswing fit into all of this?

Project Glasswing is an ambitious, multi-party vulnerability discovery and disclosure project that Anthropic is undertaking with the above-mentioned vendors and a few others. Claude Mythos discovered “thousands of zero-day vulnerabilities” in products and open-source projects, which is a pretty scary sounding statement. Part of the project involves Anthropic reporting the bugs to the affected vendors and maintainers and then somehow coordinating disclosure and patch timing. This is a non-trivial task under the best of conditions, and doing it with a massive vulnerability set across many of the largest and most critical vendors is mind-bendingly thorny. 

“This is the first time that thousands of vulnerabilities are being discovered and disclosed all at once to a number of different companies,” Katie Moussouris, CEO of Luta Security, and an expert on vulnerability disclosure, told Decipher last week. “I think we’re going to see a lot more outages, even though the release of this model is being structured in a way which is absolutely admirable in trying to do this as efficiently as possible, the open source models will catch up, our adversaries with their distillation tactics will catch up.”

Unlike a lot of things associated with AI these days, Project Glasswing appears to be an unalloyed good. 

Why isn’t Anthropic going to let the public have access to Claude Mythos?

Basically because they say it’s to protect us from Mythos’s big brain. Mythos is advanced enough that it’s producing outputs that are too dangerous from a cybersecurity point of view. In other words, Mythos is smarter than you are.

“We do not plan to make Claude Mythos Preview generally available, but our eventual goal is to enable our users to safely deploy Mythos-class models at scale—for cybersecurity purposes, but also for the myriad other benefits that such highly capable models will bring. To do so, we need to make progress in developing cybersecurity (and other) safeguards that detect and block the model’s most dangerous outputs,” Anthropic says.

If that reminds you of plot lines in a movie or two, you're not alone.

So we’re about to get a tsunami of new vulnerability reports and patches?

We sure are. The release schedule for the bugs that Claude Mythos found hasn’t been published yet, but it’s a good bet that there will be a sharp increase in the number of vulnerabilities patched by Cisco, Microsoft, Google, et al in the next few months. And that’s just the leading edge of the wave. These models will only get better at finding vulnerabilities and the humans prompting them will only get better at creating instructions for them. And as the models become more widely available, it’s very likely that we will start seeing serious and critical vulnerabilities in commercial software and open source projects that have not yet seen focused attention from top vulnerability researchers.

Elite vulnerability research talent historically has been scarce and concentrated mainly in the hands of a small number of very well-funded teams such as Google Project Zero and Microsoft Vulnerability Research, as well as individual researchers and government-backed teams. The advent of LLM-assisted bug hunting will democratize that process in a way that we haven’t seen before at scale.

Does that mean exploitation is going to spike too?

Not necessarily. If we locked the gates right now and no one ever disclosed another vulnerability, attackers would not run out of bugs to exploit anytime soon. If ever. Bugs abound, patching is difficult, and budgets are finite. And most known vulnerabilities are never exploited. 

“I do not agree with the conclusion that this suddenly translates into a significant new risk of widespread exploitation. Because absolutely none of this fundamentally changes the current threat landscape. Sure, AI will almost certainly accelerate vulnerability discovery and add to everyone’s existing backlog. But that is not the constraint. The constraint has never been vulnerability or exploit availability. It has always been selection,” Jeremiah Grossman of Root Evidence wrote.

In short, attackers are human (for now) and they have agendas and priorities too, so they pick and choose their targets accordingly. What the AI revolution does mean, though, is that offensive tools and infrastructure will be available to a wider range of attackers, which is certainly a concern.

This all sounds interesting, but I’m the only IT person in a company with 200 employees. How does this affect my security posture?

Probably very little for the foreseeable future. The proliferation of AI tools for vulnerability research and offensive operations will accelerate bug discovery and attacks, but as we said above, attackers already have plenty of known bugs to choose from (and even beyond vulnerability exploitation, plenty of alternate methods for initial access). Offensive tooling is pretty widely available already. One thing that remains true through these major technological shifts is that security fundamentals and rational thinking are vital. Understand your threat model, analyze your attack surface and identify gaps in coverage and act accordingly. Overreacting to the new new thing will do you more harm than good. To quote the classic James Mickens essay, “My point is that security people need to get their priorities straight. The ‘threat model’ section of a security paper resembles the script for a telenovela that was written by a paranoid schizophrenic: there are elaborate narratives and grand conspiracy theories, and there are heroes and villains with fantastic (yet oddly constrained) powers that necessitate a grinding battle of emotional and technical attrition. In the real world, threat models are much simpler.”

Further reading 

Cloud Security Alliance paper: The “Ai Vulnerability Storm”: Building a Mythos-ready” Security Program

Katie Moussouris's post on Mythos and bug disclosure: https://www.lutasecurity.com/post/vulnapalooza-why-anthropic-s-mythos-is-the-loudest-headliner-nobody-bought-tickets-to

Dan Geer’s essay on estimating vulnerability density: http://geer.tinho.net/fgm/fgm.geer.1504.pdf

James Mickens’s essay: This World of Ours