Mini-ML: 12M Parameter LLM Trained From Scratch in Rust

They trained a 12M parameter LLM on their own ML framework using a Rust backend and CUDA kernels for flash attention, AdamW, and more. Inspirational project for anyone who wants to better understand how to build LLMs.

Sources: tweet