A Transformer burned into custom silicon: 56K tokens/sec at 80 MHz, no GPU or CPU

A reminder that a Transformer is, at bottom, just arithmetic that can be cast directly into hardware: Fabio Guzman hand-designed a full Transformer with KV cache as a 100% digital circuit, prototyped it on an FPGA, and ran Karpathy's microGPT to spell out names at 56,000+ tokens/sec on an 80 MHz clock with no GPU or CPU anywhere in the loop.