Over the past several months, Mozilla ran an agentic harness powered by Claude Mythos Preview across Firefox's source code, identifying 271 security bugs that were fixed in Firefox 150, with additional fixes shipped in versions 149.0.2 and 150.0.1. Over 100 people contributed code to get those patches out.

The bugs spanned a wide range of subsystems. Among the disclosed reports: a 15-year-old flaw in the HTML <legend> element, a 20-year-old XSLT bug involving reentrant key() calls, a race condition over IPC that allowed a compromised content process to manipulate IndexedDB refcounts and trigger a use-after-free, and a buffer over-read during HTTPS RR and ECH parsing triggered by simulating a malicious DNS server. Several were sandbox escapes that would need to be chained with other exploits for a full Firefox compromise.

Building the Harness for Reliability

Mozilla engineers began with small-scale experiments using Claude Opus 4.6, targeting sandbox escapes specifically. Brian Grinstead, a Mozilla Distinguished Engineer, described the core requirement for making the system work at scale.

“This pipeline is extremely reliable at filtering out false positives, so long as you have a clearly defined success condition to validate against. Memory corruption issues are especially easy to validate: either you trigger Address Sanitizer or you don’t (in which case you tell the agent to keep working until it does),” Grinstead said.

False positives did occur in limited numbers. “We did see a small handful of false positives, primarily caused by changing some precondition in order to trigger an issue that would otherwise be valid (e.g., enabling a testing preference or using a private API). We’ve seen fewer than 15 of these total, and when we see them we update the harness to prevent similar issues in the future,” he added.

Once working, the team parallelized jobs across multiple ephemeral virtual machines, each assigned a specific target file. Findings were written to a bucket, then fed into a security bug lifecycle covering deduplication, triage, classification, and patch management. This architecture allowed Mozilla to scale from dozens of targets to thousands without compromising accuracy.

Containment and Safety Measures

Crafting sandbox escape proofs required the model to patch Firefox source code, a step that raised questions about containment. Grinstead explained the controls in place.

“Any source code changes made to craft sandbox escapes are only used to generate bug reports. They never land in the upstream Firefox source code, nor are they published anywhere outside of the bug tracker. Scanning happens entirely within isolated VMs that have a local copy of Firefox’s open source codebase, with no means to publish their changes. After completing the analysis, any findings are written into an internal database and the VM is destroyed,” he said.

This approach echoes standard practices in vulnerability research where exploit code is carefully quarantined. Mozilla's infrastructure also used network isolation and restricted API tokens to ensure that even if the AI generated malicious code, it could not affect production systems. The VMs were configured with minimal privileges and automatically terminated after a set period, preventing any long-lived exposure.

What the Model Could Not Break

Equally notable is what the system failed to exploit. Mozilla engineers observed the harness repeatedly attempting to escape the process sandbox via prototype pollution in the privileged parent process, a technique that had succeeded for external researchers in prior years. An architectural change to freeze those prototypes by default blocked every attempt.

This defensive design decision, made years before AI-assisted hacking became practical, proved critical. By freezing prototype objects—effectively preventing adversaries from modifying built-in JavaScript objects—Mozilla removed a common attack vector that the AI kept trying. The result highlights the importance of proactive hardening based on known vulnerability patterns.

Other areas the AI struggled with included race conditions that required precise timing across processes and vulnerabilities buried in legacy code paths not covered by the initial file selection. The harness also failed to exploit certain integer overflow bugs because the model could not correctly reason about arithmetic boundary conditions without extensive trial and error.

Vendor Accountability and Defensive Pressure

“What stood out to me in the report was how Mozilla’s past architectural decisions to apply layered defences within Firefox resulted in Mythos repeatedly failing to exploit vulnerabilities. This reinforces what many cybersecurity professionals have advocated for years that cybersecurity needs to be included in all software at the design, development, and the deployment stages,” Brian Honan, a cybersecurity and data protection expert and owner of BH Consulting, told the publication.

On vendor accountability, Honan pointed to procurement as the primary lever available to security leaders. “The core leverage CISOs have with vendors who are not doing this kind of hardening is their budget. Working closely with their company’s procurement functions CISOs should look to include security requirements as mandatory elements of purchasing decision making process. Part of those requirements should include asking vendors during a tender or renewal process what secure practises they have around the development of their product or service and whether they are incorporating AI-assisted code analysis into their secure development lifecycle,” he said.

This perspective is increasingly relevant as AI code generation tools become mainstream. Organizations that purchase third-party software must demand evidence of continuous security testing, including automated pipeline scanning. The presence of an AI-assisted bug finding system can be a key differentiator in vendor selection, shifting competitive pressure toward security maturity.

Historical Context: Why This Matters

Mozilla's experiment is part of a longer trend of using automation to find software vulnerabilities. Traditional fuzzing tools like AFL and libFuzzer have been used for years to find memory corruption bugs, but they struggle with logic errors and multi-step exploits. AI agents, by contrast, can chain multiple actions—reading source code, modifying it, compiling, and checking for crashes—in a way that mimics a human security researcher.

What sets this work apart is the scale: 271 confirmed bugs from a single AI-driven pipeline. Traditional static analysis and fuzzing combined often yield fewer actionable results in the same time frame, especially for complex browsers. Moreover, the AI found bugs that had persisted for over a decade, showing that human reviewers repeatedly missed these issues during code audits.

The use of a paid API model (Claude Mythos Preview) also introduces cost considerations. Mozilla has not disclosed the total compute cost, but the ability to run such a pipeline on commercial hardware without dedicated GPU clusters makes it accessible to medium-sized organizations. As inference costs drop, similar pipelines could become standard in security testing.

Future Integration and Industry Implications

The team plans to integrate the analysis into continuous integration to scan patches as they land, shifting from file-based to patch-based scanning. This means every code change submitted to Firefox will be automatically analyzed by the AI for potential exploits, catching vulnerabilities before they ever reach a release. This proactive approach could drastically reduce the number of zero-day bugs that make it into production.

Mozilla’s recommendation to other development teams is to begin with simple prompts and iterate, treating the harness as infrastructure that compounds value with each model generation upgrade. They emphasize that the key is not the sophistication of the initial prompt but the reliability of the validation step. A clearly defined pass/fail condition turns an imperfect AI into a powerful tool that improves over time as the underlying models improve.

For the broader industry, this project demonstrates that AI-assisted security testing is no longer theoretical. It finds real bugs that humans miss, and it can do so at a speed that outpaces traditional methods. The main barriers are no longer technical but organizational: companies must invest in building the harness, training their teams, and restructuring their build systems to accommodate automated testing without breaking developer workflows.

As AI models continue to evolve, the gap between what they can exploit and what they cannot will narrow. Mozilla's experience shows that strong architectural defenses can hold the line, but those defenses must be designed and deployed before the AI shows up to test them. The race between AI offense and defense is now squarely on the agenda of every software development organization.

Source: Help Net Security News