Sebastian Raschka · 2026-09-09 · notable
Sebastian Raschka — looped transformers and what Astra's short reasoning means
Sebastian Raschka explains weight-shared 'looped' transformers and pushes back on the idea that GPT-6 Astra hides its reasoning. He argues short chains of thought track model capability, the way bigger models always needed fewer tokens.
Sebastian Raschka walks through looped transformers and argues Astra's brief reasoning traces are capability, not concealment.
What is it?
Looped transformers are the subject of Sebastian Raschka's 9 September 2026 post, written in the week after GPT-6 Astra shipped. The article walks through weight-shared architectures — designs where the same transformer blocks process a representation several times — and connects them to the running argument about why Astra's visible reasoning is so short.
How does it work?
Weight sharing is the thread across the four variants Raschka compares. Universal Transformers choose how many passes to take with adaptive halting; Nanbeige fixes the count, sending a representation through its 22 blocks twice; Ouro runs four passes with exit gates; Mixture-of-Recursions routes per token. Raschka reads the evidence as saying looping can improve quality at a fixed compute budget, but only once the model is large enough.
Why does it matter?
The post takes on a popular claim about GPT-6 Astra — that its brief chains of thought mean reasoning is being hidden inside the architecture. Raschka argues that shorter traces track capability instead, since larger models have always needed fewer tokens than small ones to reach the same answer. He pairs that with recent work on latent reasoning and compute-matched comparisons, which gives readers a way to judge the next 'hidden reasoning' claim.
Who is it for?
engineers following LLM architecture