Vals tests show open-weight models nearly match closed AI in network hacking
Research by Vals indicates free Chinese AI models perform almost as well as restricted frontier systems in simulated cyberattacks.
Key points
- Open-weight model DeepSeek performed closely behind closed systems in simulated intrusions.
- OpenAI’s restricted Daybreak model completed 81 percent of tasks in the Vals study.
- Access restrictions may not delay the spread of advanced hacking capabilities.
Vals, an AI capability measurement firm, reported that open-weight artificial intelligence models now rival closed systems in executing complex network intrusions.reported The research tested how effectively different models performed multi-step instructions on mock corporate environments.Vals found that free AI models can now hack simulated networks almost as well as expensive, restricted ones.Vals data indicates open-weight models like DeepSeek achieve 73 percent task completion, nearing OpenAI’s 81 percent benchmark. This finding challenges the assumption that proprietary models hold a decisive advantage in cybersecurity skills.
What changed
The study evaluated models against two simulated scenarios involving mock corporate networks. One scenario required accessing a network through a public website. The other demanded compromising one company network to reach a second. These tests measured the ability to carry out specific, sequential cyber operations.
How it works
Researchers gave AI models specific tasks within these simulated environments. OpenAI’s Daybreak, a cyber-focused model available only to trusted partners, achieved the highest score. It completed 81 percent of assigned tasks, averaging two attempts per objective. This benchmark established the performance ceiling for the experiment.
What the evidence means
DeepSeek, a Chinese open-weight model, completed 73 percent of the objectives on average.reported The Vals researchers noted that on certain tasks, DeepSeek scored equally with OpenAI’s system. This suggests that open models accessible to anyone are closing the capability gap with restricted frontier systems.
Cost and market position
The findings imply that efforts by frontier labs to limit model access may be ineffective at slowing capability proliferation.
Why it matters
- Product teams can no longer assume closed models offer superior cybersecurity performance.
- Open-weight models provide near-parity capabilities without the access restrictions of frontier labs.
- Defenders must prepare for widely available advanced hacking tools rather than relying on access gates.
Topics: AI security, open-weight models, Vals, DeepSeek, network intrusion
Updated
Ask the article
Local keyword search of this article; no generated answers.Did this article give you the evidence you needed?
Your choice stays on this device. No public tally.Questions readers ask
Which AI model performed best in the Vals network intrusion tests?
OpenAI’s Daybreak model completed 81 percent of tasks, outperforming others in the study.
How did the open-weight model DeepSeek perform compared to Daybreak?
DeepSeek completed 73 percent of objectives, with equal scores on some specific tasks.
Does this mean open models are now better than closed ones?
No, Daybreak still scored highest, but the gap between open and closed models has narrowed significantly.
Connected stories
Reading receipt
- Open-weight model DeepSeek performed closely behind closed systems in simulated intrusions.
- OpenAI’s restricted Daybreak model completed 81 percent of tasks in the Vals study.
- Access restrictions may not delay the spread of advanced hacking capabilities.
Still open: How will these findings impact real-world cyber defense strategies?
Still open: Will other open-weight models achieve similar results in independent tests?
