Google TurboQuant: 6x KV-Cache Compression with Zero Accuracy Loss

Google releases TurboQuant, a compression algorithm that reduces LLM key-value cache memory by at least 6x and delivers up to 8x speedup with zero accuracy loss. The technique combines online vector q

Federico Ulfo