LLMs That Rewrite Their Own Weights—No Training Needed

Original: Language Models Can Rewrite Their Own Weights

Why This Matters

Self-modifying weights without training could change how models are deployed and updated.

Researchers demonstrate 'self-rewriting': a 27B open-weight LLM edits its own weights using reasoning alone—no gradients, no retraining—gaining 6.6 accuracy points over 8 sessions on unseen problems.

A team of 11 researchers has published a paper on arXiv showing that a language model can improve itself by directly rewriting its own weights—using only forward passes and its own reasoning, with zero gradient computation. They call the technique 'self-rewriting,' positioning the model as its own optimizer.

The setup: a 27B open-weight model is given access to its own weights, graded answers to problems, and the ability to run itself. It diagnoses where it fails, rewrites the responsible weights, then tests those edits on practice problems before committing only the ones that work. Over eight consecutive sessions, accuracy never drops—each round builds on the last, adding up to a 6.6-point gain on problems the model never saw during the process.

The paper also demonstrates a repair capability: a damaged copy of the model—one that solves almost nothing—is restored to over 80% of the original's accuracy on image tasks, without the repaired copy ever reading the original weights and without any retraining. Scale matters here; in one model family, only the largest model successfully self-edits—smaller variants either make no changes or break themselves.

The authors argue this blurs the line between training and inference, noting that unlike test-time scaling, gains are baked into the weights and cost nothing per subsequent answer. They frame self-rewriting as a step toward recursive self-improvement—a model that continuously upgrades itself with no external scaffolding.

Source

arxiv.org — Read original →