LLMs Can Now Read and Rewrite Neural Weights
Original: Astra can read and write weights to improve Qwen
Why This Matters
Weight-Space Programming could redefine how models are optimized and debugged post-training.
MIT CSAIL researchers found that GPT-6 Astra can read and edit raw neural network weights without gradients, boosting Qwen3.5-0.8B's MATH-500 score from 7.9% to 48.1% by modifying just 0.016% of weights.
A team at MIT CSAIL has published early findings showing that frontier LLMs can natively interpret and rewrite neural network weights — a task considered practically impossible for humans. They call this approach 'Weight-Space Programming.' In their tests, Astra (GPT-6) identified a reasoning-loop bug in Qwen3.5-0.8B's weights that Qwen's own model card flagged as a known issue, then fixed it by editing just 0.016% of the model's weights with no gradient-based training. The result: MATH-500 accuracy jumped from 7.9% to 48.1%, and IFEval (instruction following) tripled. Astra also wrote game-playing networks from scratch that outperformed policies trained with PPO for 100 million steps, and correctly identified what task a 17,253-weight network had been trained on with 99% confidence. The key mechanism is deliberate reasoning about what each component of a network computes or should compute — not statistical pattern-matching or brute-force search. The authors argue this signals a shift away from symbolic optimizers like Adam or Muon, toward LLMs that actively reason about weight space. No training loops were allowed in any of these experiments.