Community Updates
A stream of short-form insights, takes, and news from the AI Socratic community.
Anthropic: Natural Language Autoencoders (NLAs)
Models don't always say what they think, they instead encode their thinking into tokens that are not human readable. Anthropic introduces a solution to train models to convert internal neural activati
SakanaAI × Nvidia: Sparser, Faster, Lighter Transformer (TwELL)
Sakana AI & NVIDIA's ICML 2026 paper introduces TwELL — a new sparse format for LLM feedforward layers that achieves 95% unstructured sparsity via ReLU + light L1 while staying fully compatible with f
The First Law of Complexodynamics
Scott Aaronson asks why physical systems become more “interesting” before settling into disorder, even though entropy only increases. Using a coffee cup example separate → swirling patterns → fully mi
We know why RAG hallucinates
~97% of your vector database is mathematically empty. Your RAG system is retrieving from noise. Sources: tweethttps://x.com/anirudhbvce/status/2052532004919361958