Researchers introduce Program-of-Layers (PoLar), a training-free framework that dynamically skips, keeps, or repeats transformer layer segments per input, achieving up to 87.8% on DART-Math with Qwen2.5-3B, without modifying base model weights.
A new paper accepted at ICML 2026, "Skip a Layer or Loop It? Learning Program-of-Layers in LLMs" (PoLar) by Ziyue Li, Yang Li, and Tianyi Zhou, proposes a training-free method that treats frozen transformer layers as a callable library. Instead of every token passing through the same sequential layers, a lightweight 2.1M-parameter predictor compiles a global execution program per input, deciding for each contiguous segment (up to 4 blocks) to skip, keep, or repeat it.
On the DART-Math benchmark with Qwen2.5-3B, the results are dramatic:
Crucially, 34% of baseline failures are rectified using fewer layers than the original model depth. The overhead is negligible — 3.05ms (~0.23 of an LLM layer, <0.06% of model params) — and net wall-clock latency drops by 0.83× on simple tasks.
Trained purely on math via offline MCTS supervision, the controller transfers zero-shot to MMLU-Pro (humanities, social sciences, natural sciences) and out-of-distribution word problems, suggesting it learns generalized execution policies rather than prompt memorization.
Key limitations: supervision depends on offline MCTS search; fixed constraints (max 4-layer chunks, max 1 loop) prevent deep recursion; routing is sequence-level (tokens share the same path).
This work demonstrates that fixed-depth, static-order inference exposes only a fraction of an LLM's intrinsic reasoning capability. By dynamically compiling input-dependent layer execution paths, PoLar proves that latent reasoning can be systematically scaled and reallocated at test time without retraining or modifying any base model weights. The approach offers a better Pareto frontier of latency vs. accuracy, challenging the assumption that test-time compute scaling must happen in token space (e.g., chain-of-thought).