Visa has published the security tooling it used to hunt for vulnerabilities inside its own payment network, and buried in the documentation is a sentence that should stop any engineering manager mid-scroll: “A plain scan edits your code.” The Visa Vulnerability Agentic Harness — VVAH — does not stop at telling you what is broken. In its default configuration it writes the fix into your source files, and it does that before a single human being has looked at either the finding or the patch.
That is the part of the story worth arguing about, and it is more nuanced than a headline can carry. The tool edits a checkout on a developer’s machine or a build agent, not a live production system. Nothing it writes is merged, signed, deployed or released without somebody pressing a button.
But the sequence genuinely does run finding, fix and validation end to end with no human gate in the middle, and the first person to read the patch is reading a change that has already been applied. Visa’s own governance around the tool is explicit that people stay in the loop; the tool’s default behaviour is explicit that they are not in the middle of it.
The harness grew out of Project Glasswing, Anthropic’s defensive security initiative, and the model behind that initiative, Claude Mythos, is what gave Visa the raw capability to find bugs faster than it could fix them. This article walks through what Visa actually shipped, what the four-phase pipeline does at each stage, exactly where the “before any human reviews it” claim is true and where it is not, and what the independent evidence says about whether an AI-written security patch is worth applying at all. The short version of that last point: on the best measurement available, roughly one in four holds up.
Table of contents
- What Visa Actually Shipped
- The Visa Pipeline: Four Phases, Eleven Stages
- The Detail Behind the Headline: Visa’s Default Is Fix Mode
- What Visa Says About Human Review
- The Evidence on AI Patch Quality
- How Much Visa’s Approach Is Really Finding
- What This Means If You Run a Codebase
- Where Visa’s Release Fits in the Wider Shift
- Frequently Asked Questions
- References
What Visa Actually Shipped
The release is a harness, not a model, and that distinction explains almost everything about how it behaves.
The harness, not the model
Visa did not train anything. VVAH is an orchestration layer written in Python that drives a frontier large language model through a fixed sequence of security tasks, wraps it in deterministic controls, and emits machine-readable output at the end. The intelligence is rented; the discipline is the product. That is why Visa could open-source it without giving away a model, and it is why the repository is small enough to read in an afternoon.
Where it came from
Visa was one of roughly fifty organisations given early access to Claude Mythos Preview under Project Glasswing, alongside Amazon Web Services, Apple, Broadcom, Cisco, CrowdStrike, Google, JPMorganChase, the Linux Foundation, Microsoft, NVIDIA and Palo Alto Networks. Visa pointed the model at its own payment infrastructure, which processes hundreds of billions of transactions annually, and built increasingly structured scaffolding around it. VVAH is the fifth generation of that scaffolding.
What is in the repository
| Item | Detail |
|---|---|
| Name | Visa Vulnerability Agentic Harness (VVAH) |
| Published | 10 June 2026, github.com/visa/visa-vulnerability-agentic-harness |
| Licence | Apache License 2.0, Copyright 2026 Visa, Inc. |
| Requirements | Python 3.11 or later, plus LLM credentials |
| Generation | Fifth iteration of the internal harness |
| Pipeline | Four phases, eleven stages, plus optional stage 0 |
| Output | SARIF 2.1.0, Markdown findings, CVSS 3.1, CWE mapping |
| Model backends | Claude CLI or SDK, OpenAI-compatible endpoints, open-weight servers |
| Default profile | Runs to phase 4 fix mode and edits source files |
| Detection-only switch | --stop-after s9 |
| Build or test step | None — the harness compiles nothing |
Who is behind it
The announcement carries two names with real authority inside Visa: Rajat Taneja, President of Technology, and Subra Kumaraswamy, Chief Information Security Officer. That places the release in the operating core of the company rather than in a research group, and it is a signal about intent. Visa is not publishing a curiosity. It is publishing the thing it runs.
Why a payments company open-sources its defences
The logic is the same one that drives coordinated disclosure. Visa’s exposure is not limited to code Visa wrote; it extends to every library, gateway and integrator in the chain. A harness that only Visa can run protects Visa slowly. A harness anyone can run protects the network the company sits inside. Taneja’s framing was blunt: “In a world of agentic attacks, defense also has to be agentic.”
The Visa Pipeline: Four Phases, Eleven Stages
Understanding where the human review question lands requires knowing what runs when. The pipeline is fixed, numbered and documented, which is unusual enough in agentic tooling to be worth noticing.
Phase 1 — discovery and threat modelling (S1 to S3)
The opening phase maps the attack surface and builds what Visa calls a threat-aware plan. The agent reads the repository, works out what the software is for, identifies trust boundaries and decides where to spend effort. Nothing is claimed as a vulnerability yet.
Phase 2 — multi-lens analysis and adversarial verification (S4 to S6)
This is where candidate findings appear and, crucially, where most of them die. The harness runs several analytical passes with different framings, then turns an adversarial agent loose on the results with the job of disproving exploitability. A finding that survives a hostile reviewer is worth more than one that a single pass produced.
Phase 3 — deduplication, chaining and SARIF emission (S7 to S9)
Stage 7 through 9 deduplicate overlapping reports, chain individual weaknesses into exploit paths and emit structured output. The chaining is the piece Taneja singled out: “It can construct attack chains, showing it will find things which may not be high severity, but put them together.” Three medium-severity issues that combine into remote code execution are exactly what a signature-based scanner cannot see.
Phase 4 — remediation and adversarial validation (S10 and S11)
Stage 10 proposes candidate fixes. Stage 11 convenes what the documentation calls an agentic panel to adversarially validate those patches before adoption. In the default profile, stage 10 does not propose a patch to a person. It writes it.
Stage 0 and the static seed
There is an optional stage 0 that runs a tree-sitter call-graph analysis first, seeding the run with taint paths so the model starts from structure rather than from prose. It is off by default and it is the cheapest way to make a large repository tractable.
| Stage | What happens | Human gate in the default run? |
|---|---|---|
| S0 (optional) | Static AST and call-graph seed | No |
| S1–S3 | Discovery, threat model, research strategy | No |
| S4–S6 | Multi-lens analysis, adversarial verification | No |
| S7–S9 | Deduplicate, chain, emit SARIF | No — but this is where --stop-after lands |
| S10 | Remediation: candidate fixes written to source files | No |
| S11 | Agentic panel adversarially validates the patches | No |
| After S11 | Build, test, review, merge, deploy | Yes — entirely human, and entirely yours |
The Detail Behind the Headline: Visa's Default Is Fix Mode
Most coverage of the Visa release described a tool that finds vulnerabilities. The documentation describes a tool that fixes them unless you tell it not to, and the difference matters more than any other line in the repository.
“A plain scan edits your code”
The README says it in six words. The default profile does not stop at reporting; it continues into phase 4 fix mode, and fix mode edits source files in the target repository. An engineer who clones the harness, points it at a service and runs the obvious command gets a modified working tree. Not a pull request. Not a diff in a report. Modified files.
What “before any human reviews it” does and does not mean
It is true that no person approves the patch before it is written. It is not true that the patch reaches production. Those two statements are compatible, and collapsing them is how the story gets distorted in both directions — into a scare about autonomous deployment on one side, and into a shrug about a harmless scanner on the other.
| Claim | Accurate? | What the documentation actually supports |
|---|---|---|
| The AI writes patches with no prior human review | Yes | Fix mode is the default; S10 applies edits directly |
| The AI validates its own patches | Yes | S11 runs an adversarial validation panel of agents |
| Patches land in production automatically | No | Nothing merges, builds, tests or deploys itself |
| The harness tests its fixes | No | “VVAH performs no compilation, building or testing” |
| Visa runs it unsupervised internally | No | Visa states teams review findings and set remediation paths |
| Results are repeatable | No | Output is LLM-generated and explicitly non-deterministic |
The stop-after switch
Detection without modification is one flag: --stop-after s9. That halts the run at SARIF emission, before remediation begins. Any organisation evaluating the harness should treat this as the starting configuration and earn its way to fix mode, not the other way round.
Why the default matters more than the disclaimer
The repository carries a clear warning — “Always review the generated fixes and build/test them yourself before merging” — and Visa’s blog is equally clear. But defaults do more work than disclaimers. Most people run the obvious command, and the obvious command edits code. A tool that ships safe-by-default with an opt-in to autonomy sets a different norm than one that ships the reverse, and this one ships the reverse.
Where model choice quietly becomes a control
There is a subtlety that functions as an accidental safety valve. Full file-editing capability requires an Anthropic backend; run VVAH against an OpenAI-compatible endpoint and it is limited to report-only output. Model selection is therefore also a permissions decision, which is not how most teams think about a configuration file.
What Visa Says About Human Review
Set the tool’s defaults aside and read Visa’s own governance language, because the company was careful and specific.
The governance line
“Human oversight remains essential at every key point in the workflow. Our security and engineering teams review findings, validate severity, and determine remediation paths before changes are advanced.” The operative phrase is “before changes are advanced” — advancement, not generation, is the gate.
Kumaraswamy on force multipliers
The CISO’s framing was the more conservative of the two: “AI is not a substitute for strong security foundation, it is a force multiplier.” That is a claim about leverage rather than replacement, and it is consistent with a tool that produces candidates at volume for humans to adjudicate.
Taneja on machine speed
Taneja’s version pushes harder, describing a shift of emphasis “from finding issues to validating, prioritizing and fixing at machine-speed.” Both can be true at once: generation and validation move to machine speed, while the decision to advance a change stays at human speed. That is the operating model the harness implies, and it is a reasonable one.
The honest tension
There is still a gap between “requires human review” and a default that edits before review happens. Visa resolves it internally with process. Anyone downloading the harness inherits the default without the process, and that is the risk the release actually creates. It is the same governance gap that shows up whenever agentic tooling is handed capabilities faster than organisations build the controls around it — a pattern familiar to anyone building an agentic SOC.
The Evidence on AI Patch Quality
The strongest argument for keeping a human between an AI-written patch and a merge is not philosophical. It is measured, and the measurement is recent.
The Off-by-1 Labs study
In August 2026, Off-by-1 Labs published the largest evaluation of AI-generated security patches so far. The team, led by Keith Hoodlet, took six recently disclosed vulnerabilities — a Linux privilege escalation, an ActiveMQ remote code execution flaw, a Chrome file-system use-after-free, an EXIM RCE, a SpringAI SpEL injection and a Gemini CLI RCE — and generated 540 patches for each across three environment configurations and nine prompt templates, using two frontier models. That is 6,080 patches in total. The paper was peer reviewed by Casey Ellis, Jason Haddix and Mike Shema.
What 6,080 patches actually did
Reading the number that matters
One patch in four resolved the vulnerability without changing what the application did. Add the fixes that worked but altered behaviour and you reach 46.1% — still under half, and “altered behaviour” in a payment path is not a rounding error. The remaining 53.9% either left an exploitable route open or created a new one.
Fragile fixes and FLAWED artefacts
The subtler finding is worse than the headline rate. More than a third of the patches that were scored as successes contained what the researchers called fragility: they blocked the specific proof-of-concept input without addressing the root cause. A patch that defeats the published exploit and leaves the underlying defect intact is the most dangerous outcome in the set, because it closes the ticket. Hoodlet’s summary: “LLMs produce Fix-Like Artifacts with Embedded Defects (FLAWED) 53.9% of the time when complex patches are required.”
What generation costs
Inference ran between $2.11 and $2.81 per patch. A single 540-patch sweep against one vulnerability therefore cost between roughly $1,139 and $1,517, and the full 6,080-patch study lands near $15,000 at the midpoint rate. That is cheap enough that volume is not the constraint. Judgement is.
| Outcome | Share | Safe to merge unreviewed? |
|---|---|---|
| Complete fix, behaviour unchanged | 26.0% | Only if you can tell it apart from the rest |
| Complete fix, behaviour altered | 20.1% | No — needs regression testing |
| Exploitable path still open | 49.3% | No — and it looks fixed |
| Fixed, new vulnerability introduced | 2.3% | No — net negative |
| Not fixed, new vulnerability introduced | 2.2% | No — strictly worse than doing nothing |
How Much Visa's Approach Is Really Finding
The case for the harness rests on the discovery half of the pipeline, and there the numbers are genuinely strong.
23,019 findings and what survived
Across more than a thousand open-source projects, Glasswing scanning produced 23,019 findings at all severity levels, of which 6,202 were estimated to be high or critical. Security firms then independently assessed 1,752 of them. The true positive rate came back at 90.6%, with 1,587 confirmed valid and 1,094 — 62.4% of those assessed — confirmed as high or critical. Anthropic projects roughly 3,900 genuine high- or critical-severity vulnerabilities in the scanned open-source corpus.
A 90.6% true positive rate is the real result
Signal quality, not volume, is what makes an automated finding useful. Cloudflare reported 2,000 bugs, 400 of them high or critical, with false positive rates it judged better than human testers. Mozilla found 271 vulnerabilities while testing Firefox 150 — more than ten times what the same process surfaced in Firefox 148. The UK AI Security Institute recorded Mythos Preview as the first model to solve both of its cyber range simulations end to end.
Patching, not finding, is the bottleneck
And here is the number that explains why Visa built remediation into the harness at all. Of the high- or critical-severity bugs found in open-source projects, 530 were reported to maintainers. By publication, 75 had been patched and 65 public advisories had been issued. Average patch time ran to two weeks per bug.
The asymmetry that justifies the tool
Discovery scaled by more than an order of magnitude. Remediation did not move at all. A harness that generates candidate fixes is a rational response to that gap, even if only a quarter of what it generates is directly usable — because a reviewed, rejected patch still costs less than a blank editor. That is the honest case for the Visa design, and it does not require pretending the patches are good.
The counterweight nobody mentions
Worth holding alongside all of this: CISA data indicates fewer than 1% of CVEs are ever actively exploited. Volume of findings is not the same as volume of risk, and a pipeline that produces thousands of true positives can still bury the handful that matter. Prioritisation is doing quiet, load-bearing work in this design.
What This Means If You Run a Codebase
The harness is free, the licence is permissive and the barrier to trying it is one Python environment. That combination means it will be run inside organisations that have not thought about any of the above.
Run detection before you run remediation
Start with --stop-after s9 and live with SARIF output for a few cycles. Learn what the tool finds in your code, what proportion survives your own triage and how noisy it is against your stack. Only then consider letting it write.
Never point fix mode at a working tree you care about
Give it a clean, disposable checkout on a dedicated branch. If a run edits files, you want git diff to be the complete and reversible record of what happened. This is basic hygiene, and it is exactly what an engineer under time pressure skips.
Build and test everything it writes
The harness compiles nothing and runs no tests, which means an unverified patch can be syntactically plausible and functionally wrong. Given that a fifth of successful patches in the Off-by-1 evaluation changed application behaviour, a full regression run is not optional. Your CI pipeline is the control the harness deliberately does not provide.
Decide who owns the merge before you start
Name the person or team accountable for approving an agent-authored security change. If the answer is ambiguous, patches will merge on the authority of the tool that wrote them, which is nobody’s authority. The same discipline that governs AI red-teaming before launch applies here after it.
Expect different answers on different runs
The documentation is explicit that output is non-deterministic and that two runs may differ. Treat a clean run as evidence, not proof, and never treat the absence of a finding as an assurance of safety.
| Control | Why it exists | Cost to implement |
|---|---|---|
| Detection-only for the first cycles | Calibrates noise before granting write access | One flag |
| Disposable checkout, dedicated branch | Makes every edit visible and reversible | Minutes |
| Mandatory build and full regression run | The harness performs no testing at all | Existing CI |
| Named human approver on the merge | Prevents accountability defaulting to the tool | A policy line |
| Root-cause check on every accepted fix | Over a third of “successful” patches were fragile | Reviewer time |
| Repeat runs before declaring clean | Output is explicitly non-deterministic | Compute only |
| Backend chosen deliberately | Only Anthropic backends can edit files | Config review |
Where Visa's Release Fits in the Wider Shift
Step back from the repository and the release is a marker for something larger happening to security assurance.
From periodic testing to continuous validation
Enterprise cybersecurity testing is following the path automated software testing took a decade ago: from scheduled, human-led exercises to something that runs continuously inside the delivery pipeline. Annual penetration tests and quarterly assessments were always a sampling strategy dressed as coverage. A harness that runs on every branch is a different proposition, and financial services — where release velocity and regulatory exposure both keep rising — is a predictable place for it to land first.
The feedback loop nobody asked for
There is an uncomfortable symmetry here. A growing share of the code entering repositories is written by AI, and the volume of that code is part of why manual review cannot keep pace. Answering machine-written code with machine-written patches, validated by machine reviewers, removes humans from both ends of a loop whose failure modes are correlated. The models making the mistakes are close relatives of the models catching them, and that is a structural weakness rather than a bug in any one product.
Open source as a security strategy
Publishing the harness under Apache 2.0 was the sharpest decision Visa made. It invites scrutiny of the pipeline itself, lets defenders with no frontier-model budget run a professional-grade methodology, and quietly raises the floor across the payment ecosystem Visa depends on. Attackers gain the same tooling, but attackers with capital already had it. The asymmetry the release removes was never protecting anyone.
What to watch next
Two things will tell you whether this becomes standard practice. The first is whether the fix-mode default survives contact with the community, or whether pressure moves the tool to safe-by-default. The second is whether independent evaluations of stage 11 — the adversarial validation panel — show it catching the fragile patches the Off-by-1 study found, because that panel is the only thing standing between generation and a human reviewer’s inbox.
Frequently Asked Questions
Does the Visa harness patch production systems automatically?
No. It edits source files in a local repository checkout. It does not build, test, merge, deploy or release anything, and every one of those steps remains a human action.
Is it free to use?
The harness is free and open source under Apache License 2.0. Running it is not free — you supply the model credentials, and inference on a large repository is the real cost.
Which models does it work with?
Claude via the CLI or SDK, any OpenAI-compatible endpoint, and open-weight models served through a Chat Completions-compatible API. The default profile requires the external claude CLI, and full file-editing needs an Anthropic backend.
How is it different from a SAST scanner?
Traditional static analysis matches known vulnerability patterns. The harness reasons adversarially, verifies exploitability and chains lower-severity issues into complete attack paths, which is precisely what pattern matching cannot do. It is also non-deterministic, which pattern matching is not.
Should we let it write to our repository?
Not until you have run it in detection-only mode long enough to know its false positive rate against your own code, and not without a build, a full regression suite and a named approver on the other side.
Is a 26% success rate a reason to avoid AI patching entirely?
Not on its own. A rejected candidate patch still costs less than starting from nothing, and the discovery half of the pipeline is performing far better than the remediation half. The rate is a reason to review every patch, not a reason to ignore the tool.
References
Visa Releases Its AI-Powered Cyber Defense System to Open Source
visa/visa-vulnerability-agentic-harness on GitHub
Visa used Mythos to hunt for bugs in its own payment network, then open-sourced the harness
Visa’s AI vulnerability framework signals a new era of continuous security testing
Project Glasswing: An initial update
Project Glasswing: Securing critical software for the AI era
Off-by-1 Labs Research: AI-generated vulnerability patches require human review
Human oversight is still critical as AI patching tools miss security risks
AI-Generated Patches Fail Half the Time
Anthropic: Claude Mythos identified 10,000+ software flaws
Claude Mythos AI Finds 10,000 High-Severity Flaws in Widely Used Software
Anthropic scales Claude Mythos to critical infrastructure in 15+ countries