Nathan Barry · 2026-11-09 · notable
Weight-space programming as a linear program — 8x fewer edits than GPT-6 Astra
In short
Nathan Barry shows that GPT-6 Astra's weight edit to Qwen3.5-0.8B can be found as a linear program. Two one-minute LP solves match Astra's scores while changing 16,291 weights instead of 130,188.
Nathan Barry · 2026-11-09 · 4 sources checked
A minute of linear programming reproduces a weight edit that took an AI model eight hours of trial and error.
Key specs
| Weights changed | 16,291 vs 130,188 |
|---|
What is it?
Nathan Barry's post revisits MIT CSAIL's 'Sparks of Weight-Space Programming', in which GPT-6 Astra edited Qwen3.5-0.8B's weights to stop the model's thinking loops. Barry shows that choosing those weight changes is a linear program, and publishes the code and edits.
How does it work?
Given an edit direction, an LP solver picks a sparse set of weight changes that meets the target. Two solves of about a minute each matched Astra on GSM8K (69.6 vs 68.2), MATH-500 (51.7 vs 50.9) and IFEval (59.7 vs 61.9). With a simple heuristic direction and no LLM in the loop, the LP reached about 93% of Astra's result.
Why does it matter?
The result suggests that part of what looked like an AI doing creative weight surgery can be done by a classic optimizer, faster and with about 8x fewer changed weights. Barry also found that a steering vector added at inference time recovers nearly the whole fix, though only within a narrow range of strengths.
Who is it for?
interpretability and model-editing researchers
Try it
git clone https://github.com/nathanrs/weight-space-lp