Federico Ulfo
In my opinion this is the most fascinating emerging properties of this year from the latest AI models. New models can debug smaller models weights 🤯.
Researchers from MIT used Astra 6 to debug and change the weights of a small Qwen 3.5 0.8B model, improving MATH-500 bench from 7.9% to 48.1% and changing other behavior. It looks like the reason is that Astra was able to reduce the reasoning in places where the model already knew the answer.
In Sparks of Weight-Space Programming, published November 2, MIT CSAIL researchers Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab gave GPT-6 Astra access to other networks' weights and asked it to read, repair or construct them. Astra proposed the weight values directly, without gradient descent or training loops. Some tests allowed no tools or feedback at all; others used a restricted Python sandbox and a budget of evaluations.
They call it Weight-Space Programming. Think of it as debugging a neural network at the level of its numerical machinery.
The team started with Qwen3.5-0.8B, a small open model with a frustrating habit: it could get stuck repeating its reasoning until it exhausted the token budget. Its own model card warned about the problem.
Astra worked through practice problems and inspected the model using a Python sandbox. With a budget of 50 batches of forward-pass feedback, it traced the looping to a direction written by the MLPs in layers 15 and 19. Then it subtracted that direction from 64 output columns in each layer.
The edit changed 131,072 weights, approximately 0.016% of the model. Under the authors' sampling settings:
The authors' reported scores with thinking enabled under the same sampling settings. Chart: AI Socratic; data: Takehi et al.
A tiny edit, a large improvement. But what improved is the interesting part. Much of the gain appears to come from letting the model finish answers it could already work out. Fixing a recurring behavior can pay off across several tasks, which helps explain why an intervention developed around math also improved instruction following.
The team tested this interpretation on Gemma 3 270M, which rarely suffered the same loops. Similar editing produced almost no improvement. Removing an obstacle was effective; adding missing reasoning ability proved much harder.
Vaibhav Tulsyan reports that 576 of 579 corrected practice answers involved looping fixes. Xingyu Dang's annotated comparison, captioned “Same recipe,” adds another important number: 49.45% MATH-500 with thinking disabled, slightly above the edited model's 48.1%.
That image alone does not establish a controlled comparison. It does make the next experiment obvious: compare the weight edit with simpler prompting and decoding changes under matched budgets. If a setting change avoids the same failure, it belongs in the baseline. The engineering achievement of finding and repairing the loop remains interesting, while the benchmark jump needs that context.
There is also a concrete check of the edited artifact. The released checkpoint's model card reports that Ahsen Khaliq rebuilt and hash-verified the changed tensors, then scored the checkpoint at 51.0% on the full MATH-500 set, with 31.6% of generations hitting the token limit. That supports the checkpoint's performance. Independently discovering the same repair is a separate test.
The team trained four small networks to play simplified 10×10 grid versions of Breakout, Pong, Pac-Man and Freeway. Each had the same architecture, input channels and available actions: 17,253 weights, arranged the same way. Their learned weight values carried the difference.
Given those values and the input/action layout, Astra identified all four games without tools or a list of possible answers. For Breakout, the structure included clues such as weights concentrated along the bottom row of the agent channel, where a paddle would sit. It also identified whether small diffusion models generated circles, triangles or squares.
The language-model experiment went further. With input and output layers removed, Astra used the middle transformer blocks to distinguish models trained on Python, romaji and English, and identified copying heads that help repeat previously seen text. Python analysis was allowed, but running the networks was prohibited.
Does that make weights readable? Parts of them, in these setups. There is a useful limit in the same study: Astra could not identify a sycophantic model from weights alone. It needed attention patterns from actual inputs. Recovering a meaningful structure is promising; explaining every behavior from a matrix of numbers remains a much bigger challenge.
A car-control network kept driving into a ramp without enough runway. Astra received its weights, a scene description and a recorded episode. With no tools or training loop, it changed about 3.5% of the weights so the car backed up before accelerating.
The edited policy succeeded in all 1,024 evaluation episodes.
The car-control repair: the edited policy gains runway before accelerating. Animation: Takehi et al., MIT CSAIL.
This is an unusually tangible example of model repair: describe what is going wrong, locate a way to change the computation, and write the fix into the network.
Astra also produced complete weights for small Breakout, Seaquest and Asterix policies in a single reply, using the supplied rules and source code. No tools, training or playing the games before evaluation. All three beat the reported PPO baselines trained for 100 million steps. Against 500-million-step baselines, Breakout scored much higher, Asterix was close, and Seaquest remained lower.
For 9×9 Go, it constructed a network of roughly 1.5 million parameters in a Python sandbox, encoding concepts such as liberties, threatened groups and captures through successive layers. Feedback mattered: with no forward passes it beat smaller PPO opponents more than half the time; with 50 it beat an equal-sized PPO network trained on 20 million games.
Alex Krentsel reports reproducing the Breakout experiment across model generations and describes a sharp improvement in the newest models. His thread also makes the scale clear: the policy contains 20,486 parameters. A sudden jump on that test is worth investigating, without assuming the same threshold applies to every network or domain.
The practical appeal is precise model maintenance. If a recurring failure can be diagnosed and repaired once, the benefit could extend across many tasks. Internal inspection might help explain why an agent gets stuck, why it ignores a constraint, or when a request needs clarification. Direct edits could become one option alongside prompting, inference settings and training.
Cruxes' Reading a Model Inside offers a useful comparison. It examines activations, the temporary signals inside a model during inference. On When2Call, a trained internal readout improved Gemma-3-12B-it's balanced accuracy for deciding when to ask for information from 0.617 to 0.704. None of its tested edits beat the readout. Changing eight neurons could alter how often the model asked, without establishing that it asked at the right moments. The broader findings are self-registered and awaiting independent recomputation.
That suggests a practical question: when should we change the model, and when should we read an internal signal and use it to route a request or trigger an external check?
The public logs include prompts, responses, reasoning summaries, sandbox code and evaluation artifacts for the reported experiments. Earlier prompt trials are excluded, and reasoning effort varied across experiments. The restrictions matter: the Qwen repair used practice data and evaluations, even though it used no gradient descent. A small edit does not automatically mean a cheap discovery process.
Scaling is still open. Larger models have many interacting computations, and their failures may depend on long conversations or particular tool outputs. Finding a reproducible bug, validating the explanation, and checking that a repair does not damage other behavior could be as demanding as writing the edit itself. Changing very few weights does not guarantee a narrow behavioral effect.
Still, the core demonstration is striking: a reasoning model can sometimes treat another network's weights as something it can inspect and deliberately modify. The next step is to find out how often that works, where it beats simpler alternatives, and whether those targeted repairs remain reliable outside the laboratory.
MIT researchCruxes reportVaibhav TulsyanXingyu DangAlex KrentselPublic experiment logs and harnessEdited Qwen checkpoint and reproduction
