DeepSeek mHC: Manifold-Constrained Hyper-Connections

DeepSeek 🐋 mHC: Manifold-Constrained Hyper-Connections This is a new banger paper from DeepSeek! Traditional residual connections e.g., in ResNets and Transformers add the layer output to the input,

Federico Ulfo