Back to NewsKimi: Attention ResidualsMarch 30, 2026Research#Kimi: Attention Residuals A more efficient way to reuse past information across layers without slowing models down. Sources: tweetRelatedExclusive Self-Attention (XSA): Two-Line Change Improving Transformers13+ Attention Mechanisms You Should KnowMoonshot Releases Kimi 2.6PrevLeCun's Team Releases LeWorldModel: End-to-End JEPA from PixelsNextTinyLoRA: Fine-Tuning 8B Models by Tweaking Just 13 Parameters
