A new set of benchmark results suggests that one of the most-watched transitions in cybersecurity has just happened: open-weight Chinese AI models, free to download and run on private hardware, are now almost as capable of hacking into computer networks as the best closed frontier models. The clearest signal came from Chinese AI models, whose performance on simulated corporate intrusion tests closed most of the gap that security labs assumed they still had years to defend.
Vals, a company that measures artificial intelligence capabilities, put a long list of models through multi-step offensive tasks on mock corporate networks. The results land at a moment when frontier labs have been restricting access to their strongest cyber models precisely because they feared what those capabilities could do in the open. If Chinese AI models can now approach those gated systems at a fraction of the cost, the entire strategy of buying time through access control needs to be re-examined. This article walks through the benchmark, the scores, the cost gap, the policy bets that preceded this moment, and what defenders should do about it.
Table of Contents
- What the Vals Cyber Benchmark Actually Tested
- Daybreak vs DeepSeek: The Scores That Matter
- Chinese AI Models and Abliteration: Terms to Know
- The Cost Gap Is the Real Headline
- Why Frontier Labs Gated Their Best Models
- Chinese AI Models Caught Up Right on Schedule
- What Real Attackers Actually Use Today
- The Cat-and-Mouse Race Behind the Scores
- The Open-Weight Defender Playbook
- What to Watch Next
What the Vals Cyber Benchmark Actually Tested
Two mock networks, one uncomfortable question
Vals set out to answer a question that cybersecurity has been circling for two years: how much capability is actually locked behind the doors of frontier labs? Its researchers, testing a field now led by Chinese AI models, built simulated environments that imitate the texture of real corporate networks. In the first scenario, the model faces a mock corporate network reachable through a public-facing website, the kind of perimeter every company has lived behind for decades. The model must chain together reconnaissance, exploitation and follow-on steps without someone steering it through each command.
The second scenario raises the difficulty deliberately. Two different company networks exist, and the model cannot reach the second one directly. It must first compromise the first network, pivot through it, and only then pursue its objectives in the second. This is the shape of real intrusion work, where the value of any single step depends on everything that comes after it.
Why these tests deserve attention
Benchmarks of this kind matter because they replace vibes with scores. Vals ran models repeatedly, averaged the results, and published a leaderboard for cyber offense that can be checked, reproduced and argued with. That rigor is why the numbers below landed hard: the gap between gated frontier capability and free Chinese AI model capability was never supposed to close this quickly, and certainly not first with Chinese AI models.
The methodology also sidesteps the chatbot safety debate. The test measures what a model can accomplish when given specific network instructions, not whether it volunteers harmful text. For defenders of the realm of Chinese AI models and their closed rivals, accomplishment is the only currency that counts.
As our profile of Kimi K3.1 Pro, the 1M-token open-weight flagship showed, the pace of Chinese open-weight releases alone would be remarkable even without the security implications.Daybreak vs DeepSeek: The Scores That Matter
The scoreboard: 81 versus 73
The most capable model in the entire benchmark was Daybreak, OpenAI’s most advanced cyber model, available only to trusted partners through gated access programs. Across the average of two test runs, Daybreak completed 81 percent of its assigned tasks. Close behind came DeepSeek, the Chinese open-weight model that anyone can download: 73 percent of objectives completed on average. On some individual tasks, the two models scored equally.
That last detail deserves emphasis: Chinese AI models are not trailing by a mile. Equal scores on individual tasks mean this is not a story about some distant chase; it is a story about parity that already exists in places. The Vals researchers explicitly noted that the open-weight Chinese AI models matched the closed champion on particular objectives, which makes the remaining gap feel more like a shrinking margin than a moat.
Reading the benchmark table
| Model | Access | Tasks completed (avg) | Approx. cost per attack |
|---|---|---|---|
| OpenAI Daybreak | Trusted partners only | 81% | $17–$46 (GPT-6-Sol tier) |
| DeepSeek (open-weight) | Anyone, free download | 73% | $2–$3 |
| Some shared tasks | Both available | Tied scores | Different by 10x |
A closed model retaining an eight-point lead is not nothing. In hands where reliability under ambiguity decides success, that margin could matter. But an eight-point average gap after seven months of Chinese AI model progress is a very different world than the multi-year head start the frontier labs were budgeting for, and the direction of travel, with Chinese AI models closing fast, is unmistakable.
Open-Weight and Abliterated: Terms to Know
What “open-weight” actually gives you
An open-weight Chinese AI model ships its trained parameters for anyone to download. You can run it on your own servers, fine-tune it, wrap it in your own tools, and — critically for the cyber context — keep every action private, because nothing passes through a provider’s logging pipeline. There is no usage policy in force at the moment your GPU does the work, because there is no usage policy enforcement point to pass through.
Abliterated models remove the last guardrails
The situation gets sharper with what the industry calls abliterated models, and Chinese AI models are its most common raw material: variants where safety training has been stripped out by modifying the weights. As Alex Zenla of the cloud security firm Edera noted, open-weight models are already “capable of producing advanced attacks and vulnerabilities today, especially when you consider the abliterated models that disable safety training by modifying weights.” Those variants of Chinese AI models circulate on the same code-sharing sites as their untampered siblings.
For a criminal operator, a free abliterated Chinese AI model with no logging, no rate limits and no abuse detection is close to an ideal tool profile. The frontier labs understood this when they chose to gate their strongest cyber models. The open ecosystem simply moved faster than the gating strategy expected.
Regulators noticed too early: Moonshot reduced critical cyber capabilities in Kimi after AI hacking fears, one of the first public demonstrations of capability rollback in response to offense potential.The Cost Gap Is the Real Headline
The token math
Vals put money on the numbers too, and the contrast is startling. The company says a single end-to-end attack run costs between $17 and $46 in tokens when executed with GPT-6-Sol, the closed tier in question. The same kind of operation with DeepSeek comes out at $2 to $3. That is roughly a tenfold difference — Chinese AI models versus the closed champion — for output quality that differs by single-digit percentage points.
Why cheap changes the attacker population
Cost is not a side note; cost is access. When an operation costs forty dollars and requires a trusted-access relationship, the population of capable operators is small, vetted and traceable. When it costs three dollars and runs on hardware you already own, the population is everyone with a GPU and a goal. That is the mechanism by which Chinese AI models widen the entire arena without ever matching the closed champion on a leaderboard.
Vals CEO Rayan Krishnan added a twist that should appeal to the defensive side: the same economics create a strong incentive for defenders to use open models rather than proprietary ones. Blue teams can run the same scanning and exploitation sweeps on their own infrastructure, at their own scale, with their own visibility — something a gated model with usage caps and partner requirements cannot offer.
The competitive backdrop is already fierce: Mistral’s trillion-parameter Le Chonk tried to leapfrog China on the open-weight index, and Chinese labs answered with more aggressive openness, not less.Cost per full attack run, measured by Vals (US dollars)
Why Frontier Labs Gated Their Best Models
The bet frontier labs made in public
None of this is a surprise to the frontier labs, because they said it out loud months ago. Anthropic premised the limited release of new models, wary of Chinese AI models on the open web — beginning with Claude Mythos Preview in April — on the idea that gated access would buy time: a head start for defenders before advanced hacking capability went broad, Chinese AI models included.
The company’s Project Glasswing gives trusted partners access to the full capabilities of new models, and frontier red team lead Logan Graham told WIRED in April that its purpose was getting those tools “in the hands of defenders to give a head start.” Graham was explicit about the clock: “We need to prepare now for a world where these capabilities are broadly available in 6, 12, 24 months. Many things would be different about security.”
Gating as a strategy, not a firewall
The framing matters. Gating was never sold as a permanent wall; it was sold as a timesale on the inevitable. Trusted-access programs, partner vetting and staged releases were instruments for pacing the rollout — a way to make sure defensive tooling had the first lap of the race. The strategy’s success therefore depends entirely on how long the lap takes. If open models finish it in seven months rather than two years, the strategy technically executed and still lost its purpose.
Chinese AI Models Arrived on the Predicted Timeline
The prophecy, and its deadline
Seven months after the Claude Mythos Preview release and the 6/12/24-month warning, the Vals results suggest the transition that Chinese AI models delivered arrived on schedule — on the near end of the schedule. Chinese AI models, and DeepSeek in particular, are performing within single-digit points of the most advanced closed cyber model on multi-step network intrusion benchmarks.
This is the precise scenario for which gated releases were designed, arriving from Chinese AI models roughly when predicted. The policy question has flipped from “what do we do if Chinese AI models catch up?” to “what do we do now that they have?” Defenders who built their 2026 roadmaps around a slower Chinese AI model curve on a two-year head start are now working with an inverted timeline.
What “on schedule” implies for the labs
If the labs’ own forecasts were this accurate, their next forecasts deserve attention too. Several researchers involved in evaluations argue the closed labs still hold deeper benches — models trained but unreleased — and intend to keep leapfrogging the open frontier continuously. On that view, gating was only ever the first tool, and the real defense is permanent overmatch: finding and patching vulnerabilities before any open-weight Chinese AI model gets to them first.
What Real Attackers Actually Use Today
Attackers are not early adopters
Here is the grounded counterweight to the benchmark headlines. Real-world observation of cybercriminal and state-backed activity in recent months shows attackers doing what attackers have always done: conserving resources and making do with older, less capable models — both proprietary and open — for most of their work. For now, the frontier of capability and the floor of everyday crime remain different places, even as Chinese AI models blur the line.
But assume the worst-funded-well
The caveat cuts the other way for the most dangerous actors. Even with strict controls, defenders must assume that the best-funded and most highly motivated attackers have already had at least some access to closed models all this time. Targeted espionage groups rarely wait for a public download; capability leakage, rented access and homegrown fine-tunes have been available to patient operators for a while. For that population, the open-weight news is not an upgrade — it is deniability and scale.
The practical effect of Chinese AI models finally catching up is therefore less about arming a new class of genius hacker and more about industrializing the middle: mid-sophistication groups that were priced out of agentic intrusion work can now run it at volume. That is where defender bandwidth will be tested next.
For background on the company behind the model in the headline, see our explainer on DeepSeek, the Shanghai-based lab whose open releases keep resetting industry expectations.The Cat-and-Mouse Race Behind the Scores
The cat-and-mouse thesis
Alex Zenla, cofounder and chief technology officer of Edera, agrees Chinese AI models are genuinely dangerous now — and adds the caveat that should frame every reading of the benchmark. The closed labs have likely trained additional, unreleased models exceeding these capabilities, he argues, so the visible leaderboard, topped by closed models and chased by Chinese AI models, is only the visible part of the race.
“I would put this very much like a cat-and-mouse game,” Zenla said. “The labs see themselves as moving fast to outpace the open-weight models, to be able to secure vulnerabilities before the open-weight models find them. Unfortunately, I do think this may be the beginning of a dangerous security landscape.”
Who is the mouse?
The unsettling part of the cat-and-mouse framing is that the mouse population is multiplying while the cats compete commercially. Open-weight releases continue regardless of what any lab decides, and every generation of Chinese AI models arrives faster than the last. The labs’ overmatch strategy is rational, but it requires permanently outrunning an ecosystem that has no coordination problem to solve and no board meeting to cancel the race.
The Open-Weight Defender Playbook
Use the open models yourself
If the capability gap to Chinese AI models is closing and the cost gap is tenfold, the defensive logic follows quickly. Vals’ own CEO Rayan Krishnan expects defenders to adopt open models rather than rely solely on proprietary ones, and the pattern is already visible: security firms fine-tuning open-weight Chinese AI models for their own hunting pipelines, then running Chinese AI models continuously on their own infrastructure where every action is logged and auditable.
That shift pairs well with the broader automation wave. The same AI machinery now powering offensive agents is behind the explosion of vulnerability discovery — as we covered when OpenAI pitched its new cybersecurity model as a safer way to hunt zero-days, agentic scanning has become a numbers game that favors whoever can run more sweeps, more cheaply, more privately.
Harden the boring things
While the model race dominates headlines, the practical defenses remain the old ones, executed relentlessly: multi-factor authentication everywhere, credential hygiene, fast patching of known-flawed software, and network segmentation that assumes the first foothold will happen. Agentic attackers amplify whatever entry exists; removing easy entries is still the cheapest defense on the market.
What to Watch Next
The next twelve months
Three things deserve watching. First, whether Chinese AI model scores hold up as tasks get longer-horizon: parity on multi-day intrusion chains, not single sessions, would signal true substitution of closed models by Chinese AI models. Second, the abliterated ecosystem — how widely modified Chinese AI model variants circulate, and how quickly their abuse becomes visible in incident reports rather than benchmark posts.
Third, watch the labs’ release choreography. If frontier labs begin keeping cyber-capable models gated indefinitely, or split “safe” commercial versions from suppressed security-capable versions, as Chinese AI models no longer force the issue as standard practice, that is the clearest sign they consider the open-weight era fully arrived. The gating play assumed a slow river; Chinese AI models just proved how fast it flows.
The bottom line
The era of defensive advantage-by-access-control is ending. What replaces it will be decided in the next few benchmark cycles — and by how fast defenders, who now have the same cheap open models available to them, choose to point them at their own networks first.
References
Sources and further reading:
- WIRED: Free Chinese AI Models Are Getting Scary Good at Hacking
- Vals Cyber Offense Benchmark
- Vals AI Capability Measurements
- WIRED: Z.ai’s Open-Weight Models Release and the Cybersecurity Question
- WIRED: Anthropic’s Limited-Release Strategy for Claude Mythos
- WIRED: Anthropic’s Mythos Will Force a Cybersecurity Reckoning
More AI coverage: explore Progressive Robot's AI Models, Tools & Releases hub — hands-on reviews, setup guides and benchmarks in one place.