Google DeepMind has published a paper with a one-word title, Recirculation, describing a trick that costs nothing to train: take the activations from deep inside a transformer, leak a small slice of them back down into a shallow layer at the next input step, and a frozen off-the-shelf model starts behaving like a recurrent network. Michael C. Mozer, Shoaib Ahmed Siddiqui, Danny Sawyer, Rosanne Liu (all Google DeepMind) and Sunny Sanyal (UT Austin, as a DeepMind student researcher) report perplexity reductions across ten language-modeling datasets and better downstream accuracy on Gemma3 — without touching a single weight.
The motivating failure is state tracking. A feedforward transformer can update its internal state only as many times as it has layers, and the layer where a state becomes usable is deep. The paper's running example, borrowed from Lepori et al. (2025): tell a model you are holding a fishing rod, say "I see a bank," then ask whether it has an ATM. The deep layers have already disambiguated bank as a riverbank, but the shallow layers processing the next question never see that, so the model answers from the ambiguous embedding and says yes. Lepori's team fixed it by hand-patching a deep activation down to a shallow layer on one critical token, cutting contextualization errors by 60%.
Figure 2: (a) state that resolves too deep in the stack is unavailable to later processing; (b) recirculation pushes it back down. Credit: Mozer et al., "Recirculation".
Recirculation generalizes that patch into something undifferentiated and permanent: at every token, mix a fraction α of a renormalized source-layer activation into a destination layer one step later. The authors use a convex mixture (α = 0.15 in the main perplexity runs) and rescale the source to the destination's L2 norm. Optimal source/destination pairs found by grid search were {11, 4} for Gemma3 1B, {18, 9} for 4B and {35, 16} for 12B — consistently mid-stack.
Crucially this is not looping. A looped transformer recurs only in depth; recirculation recurs in depth and step, which is what lets the same layer hold both z(t) and z(t+1) and therefore track arbitrary state. The paper's own head-to-head sweep finds training-free looping produces no robust benefit on Gemma3, while recirculation does across all three sizes.
Figure 8: recirculation (top) versus training-free looping (bottom) across three Gemma3 sizes. Credit: Mozer et al., "Recirculation".
Perplexity drops on nine of ten datasets, with the effect growing with model size:
| Dataset | Gemma3 1B | Gemma3 4B | Gemma3 12B |
|---|---|---|---|
| arxiv | 13.99% | 11.26% | 25.20% |
| pg19 | 14.41% | 15.72% | 35.40% |
| booksum/book | 15.95% | 15.95% | 32.91% |
| lambada | −0.72% | 0.53% | −2.81% |
The headline GSM8k figure needs a footnote. The abstract advertises "a 21% increase in accuracy on GSM8k," and the number has been repeated on X as "+21% on GSM8k pass@1" — Dan McAteer called the paper "the AI research paper of 2026" on that basis. The results section is narrower: adaptive recirculation on Gemma3 4B yields 8.8% and 20.9% reductions in error rate at pass@1 and pass@128 respectively. Those are relative error reductions, not percentage points, and the larger one is the 128-sample figure. Absolute accuracies appear only in a bar chart.
Adaptive recirculation is the paper's second contribution: an MLP that emits token-conditional, vector-valued mixture coefficients while the LLM stays frozen. It reaches a mean 23.0% perplexity reduction on Gemma3 1B, versus 8.5% for fixed recirculation, and beats full fine-tuning of the same model (21.6%). On the eight multiple-choice benchmarks, gains are modest and depend heavily on which dataset trained the MLP; training on ARC produces significant drops.
The v1 abstract claimed a 23% perplexity reduction; v2, revised ten days later, replaced that with "a systematic reduction in perplexity" and added methodological detail "which help explain the large reductions in perplexity observed." Section 4.4.1 is blunt about why: across five model families — Gemma3, Ministral3, Pythia, Qwen3 and Phi2 — all show the same qualitative sweet spot, but Gemma's perplexity gain is about 5% versus under 0.5% elsewhere, a gap the authors attribute to a Gemma-specific artifact that, once removed, brings Gemma in line with everyone else.
Figure 7: the same mid-stack sweet spot appears in five model families. Credit: Mozer et al., "Recirculation".
The deployment cost is real too. Generation is nearly free — two stacks run in parallel, which modern hardware absorbs — but prefill must be processed token by token, which the authors concede "can be quite slow" and "may be infeasible" for large-context frontier models. Optimal hyperparameters are domain-dependent, and on the 12B contextualization benchmark recirculation hurt two of three question types.
The authors decline to sell this as a product: recirculation is "not a 'shovel ready' technique" but a "methodological or philosophical contribution." The argument is about model-design affordances — instead of proposing an arbitrary architectural change and paying to train it, ask a trained network where it already wants recurrence, then amplify that. Their evidence that the framing works is that the training-free finding transferred cleanly into the cheapest possible fine-tune. It remains a preprint, unreplicated outside the authors' own runs, and its strongest number is a relative error reduction over 128 samples on one 4B model.
