Back to News13+ Attention Mechanisms You Should KnowApril 21, 2026ResearchIn the last year we've seen different types of attention arising. This blog post shows you 13 attention mechanisms you should know and the papers that discuss them. Sources: tweet, blog postRelatedKimi: Attention ResidualsExclusive Self-Attention (XSA): Two-Line Change Improving TransformersWe know why RAG hallucinatesPrevMini-ML: 12M Parameter LLM Trained From Scratch in RustNextAll LLMs Are Either Claude-Like or GPT-Like
