Chinese Open AI Models Near Top Closed Ones in Hacking Tests
Original: Free Chinese AI Models Are Getting Scary Good at Hacking
Why This Matters
Open-weight models matching closed ones erodes labs' gating strategy for dangerous cyber capabilities.
Vals AI research shows DeepSeek completed 73% of simulated network attack tasks vs. 81% for OpenAI's restricted Daybreak model, with open models costing 10–20x less per attack run.
A study by Vals, an AI capability measurement firm, tested multiple AI models on simulated corporate network intrusions and found open-weight Chinese models closing fast on the best closed frontier systems. OpenAI's Daybreak—available only to vetted partners—topped the rankings, completing 81% of assigned tasks across two mock network scenarios. DeepSeek, freely downloadable, hit 73%, with the two models scoring identically on some individual tasks. GLM also performed competitively.
The cost gap is striking. Running GPT-6-Sol through an attack scenario costs $17–$46 in API tokens; DeepSeek costs $2–$3 for the same workload, partly because open-weight models can run on local hardware. Vals CEO Rayan Krishnan says that pricing differential gives defenders a real incentive to use open models.
The findings put pressure on a key assumption frontier labs made when limiting access to powerful models. Anthropic restricted Claude Mythos Preview to trusted users starting in April, arguing gated access would give defenders a head start. Anthropic's Logan Graham told WIRED at the time: "We need to prepare now for a world where these capabilities are broadly available in 6, 12, 24 months." Seven months on, Vals' data suggests that window has closed ahead of schedule.