Alibaba's Scroll drops context compaction and beats the best long-horizon agent by 37.4 points

Alibaba Group researchers Yin Lin, Erkang Zhu, Bolin Ding and Jingren Zhou, with Columbia University's Elaine Ang, have released Scroll, a context manager that takes the opposite bet from every production coding agent: instead of compacting an agent's history when the window fills, it compresses nothing and makes the model write Python to fetch back whatever it needs. With Qwen3.8-Max as the backbone, the paper reports 94.8% on LongMemEval_S, 73.1% on BEAM_10M — 5.1 points above the best published memory system — and 86.7% on LOCA_256K, 37.4 points above the best published long-horizon agent.

Traditional harnesses truncate or summarize the serialized prompt; Scroll keeps the session in a persistent, executable environment and lets model-written code decide what gets printed into the next call. Credit: Lin et al., arXiv:2608.21690.

#Context management as a program the model writes

Both dominant approaches — compaction, which the authors note production agents such as Claude Code, Codex CLI and Cursor reportedly do as the window fills, and external memory stores — commit to what survives before anyone knows what will be asked. "Both are fundamentally lossy in the same way," the authors write: the agent sees history only through the summary or the store, "even if the raw log still exists on disk."

Scroll instead treats a session as an executable Session Environment. An append-only Event Log in SQLite gives every turn an immutable seq address, searched with BM25 rather than embeddings because that is deterministic and needs no index-time model calls; large tool payloads move to files behind lazy handles. A sandboxed, persistent Python kernel keeps a typed namespace alive across model calls, so tool outputs bind to variables instead of being pasted into the prompt. The model works through code: exec locates records (ms.search), materializes them (ms.expand) and computes over resident variables, and only what it explicitly prints crosses into the next call's working view. The kernel is fail-closed — the log is read-only from inside it, and database, filesystem, network and tool access are limited to capabilities the harness declares.

When the view exceeds budget, Scroll evicts rather than compacts: payloads fold into seq pointers first, then whole spans leave, but everything removed stays verbatim and addressable. What keeps eviction navigable is an eviction index of short headlines the model writes with each response (task, verified state, next action, status), bound to exact log addresses and rolled up in tiers — fine anchors for recent history, coarse ranges for distant history.

#The numbers, and their asterisks

On LOCA, where four agents share the Qwen3.8-Max backbone and toolset and differ only in context strategy:

That is the cleanest result: at 128K Scroll ties a plain CodeAct agent, and the gain is in not falling apart at 256K, where summarizing and retrieving lose more than 21 points. The headline 37.4-point margin is a looser comparison — the best published LOCA result at 256K is MiniMax M3 with ReAct at 49.3, on a different backbone, taken from the leaderboard.

The memory comparison carries the same asterisk, which the authors flag: they do not reproduce baselines, because "independent reproductions in this area have repeatedly led to disagreement over evaluation setup," and instead quote each system's best public number under its own backbone and judge. On that accounting Exabase M-1 still leads LongMemEval_S at 96.4 to Scroll's 94.8; the unambiguous win is BEAM_10M, 73.1 against 68.0.

#Where it loses

The ablations are the most useful part. Summarizing history at ingestion instead of keeping the Event Log collapses BEAM_10M to 19.9 overall, near zero wherever an answer needs exact historical values. Exposing the memory surface as ordinary tool calls with no persistent kernel costs 7.3 points, concentrated in instruction following (97.5 vs. 76.3). Removing the eviction index costs 1.8 points overall but 14 on preference following.

By category, Scroll leads where a few exact records must be located and reconciled — knowledge update 92.5, contradiction resolution 88.1 — and trails where the graded answer is itself a digest: summarization 70.5 against Exabase M-1's 91.9. Ingestion-heavy pipelines build that digest at write time; Scroll rebuilds it per query.

Backbone dependence is also sharper than the memory benchmarks suggest. With harness, tools and prompts fixed, LongMemEval_S spans six points across six models (94.8 down to 88.8 for the open-weight Qwen3.6-35B-A3B), but LOCA_256K spans 64 points (86.7 down to 22.7). Every backbone follows the protocol; weaker ones make more execution errors or quit early.

#Why it matters

Nearly every shipping agent harness handles a full window by throwing information away. Scroll argues that is a bug in timing rather than a necessary tradeoff: keep the log lossless, spend the model's coding ability at query time, and make compaction a view instead of a rewrite. It is not obviously expensive to run — ingestion adds no LLM calls, and median per-task input on BEAM_10M is 105K tokens, about 1% of the corpus, though the authors report tokens rather than latency or dollars. The evidence is a technical report: author-reported, one run per task, no peer review, baselines lifted from other papers rather than re-run. But the implementation is public on a scroll-research branch of a QwenPaw fork, so the claims are checkable.