AI That Finds and Fixes Vulnerabilities: CodeMender, Mythos, Codex Security and How to Judge Them

Between late 2025 and late 2026, finding vulnerabilities with AI went from research demos to programmes that report thousands of bugs. Interviewers at security teams, AI labs and tooling vendors now ask about it directly: how these systems work, what their numbers mean, and how you would use one without making things worse.

What shipped, briefly (checked November 2026)

• Google DeepMind CodeMender (October 2025): proposes patches and proactive rewrites, validated by analysis, fuzzing, differential testing and a critique agent, with human review before upstreaming. Google later paired it with cyber-tuned Gemini 3.5 Flash Cyber (July 2026) and Gemini 3.8 Flash Cyber (September 2026) for vetted partners through its Fairwind Program.
• Google Big Sleep: variant analysis from previously fixed bugs; its first real find was in SQLite in 2024.
• Anthropic Claude Mythos Preview and Project Glasswing (April 2026): gated access for defenders; by June partners had reported more than 10,000 high or critical findings, and Anthropic said verification and patching were now the bottleneck.
• OpenAI Aardvark, later Codex Security (from October 2025): threat modelling, commit scanning, sandboxed validation and patch proposals for human review.
• Microsoft MDASH (May 2026): over a hundred agents across scanning, validation by debate, deduplication and proof with triggering inputs.
• DARPA AI Cyber Challenge finals (August 2025): competing systems found 54 of 63 planted vulnerabilities and patched 43.

How discovery works

The design choice that separates useful systems from noisy ones is the oracle: something independent of the model that says yes or no. For memory-safety bugs in C and C++, AddressSanitizer gives an almost binary answer on a concrete input. For authorisation and business-logic bugs there is no such oracle, which is why results are much stronger for memory corruption than for web application logic.

Why remediation is harder than discovery

A patch that makes a scanner rule stop firing is not necessarily a fix. Common wrong patches block the one input from the proof of concept, escape data at the wrong layer, or swallow the error that revealed the bug. A March 2026 study (PVBench) found that over 40 percent of AI patches that passed basic tests failed stronger proof-of-concept testing. Vendors converge on the same safeguard: Google reviews CodeMender patches before upstreaming them, and Anthropic says every Claude Security patch must be approved by a human.

The numbers, read carefully

Benchmark and evaluation figures are mostly vendor-reported, measured with different harnesses, budgets and attempt counts, and sometimes with safeguards off. Ask what the task set was, how many attempts were allowed, how much compute, and whether the tasks could be in training data before comparing anything. The UK AI Security Institute's April 2026 evaluation of Mythos Preview is a good model of careful reporting: it gave success rates alongside the conditions (no active defenders, results rising with token budget).

What interviewers ask now

• How does an agent prove a vulnerability is real, and which bug classes are hard to prove?
• A tool sends a maintainer 140 reports in a week. How should they triage?
• How would you evaluate an auto-remediation tool before letting it open pull requests?
• Why are the most capable cyber models released only to vetted organisations, and what does joining such a programme commit you to?
• What should a security programme change if exploits get cheaper to develop?

Where to go deeper

Start with how AI vulnerability-discovery agents work, then evaluating AI auto-remediation tools. For the strategic angle read defending when exploits get cheap, for evaluation method measuring a model's cyber capability, and for governance access tiers for cyber-capable models.