Continuous Autoregressive Language Models (CALM)

Tencent + Tsinghua just dropped a paper called Continuous Autoregressive Language Models (CALM) and it basically kills the “next-token” paradigm every LLM is built on.

Instead of predicting one token at a time, CALM predicts continuous vectors that represent multiple tokens at once.

Meaning: the model doesn’t think “word by word”… it thinks in ideas per step.

→ 4× fewer prediction steps (each vector = ~4 tokens)
→ 44% less training compute
→ No discrete vocabulary pure continuous reasoning
→ New metric (BrierLM) replaces perplexity entirely

link: x.com/rryssf_/status/1985646517689208919