Skip to main content
AI Socratic
Kimi: Attention Residuals

Kimi: Attention Residuals

A more efficient way to reuse past information across layers without slowing models down.

Attention Residuals

Sources: tweet