Tarun Chitra, who just happened to be my former boss at Gauntlet, published an in depth Twitter article essay arguing that multi-model routing marketplaces like OpenRouter are vulnerable to a specific form of gaming: providers under-report cache hit rates to win ranking slots, then monetize at a higher effective price.
A Provider under reported cache hits and won the ranking. Every honest provider now has two choices: lose the traffic or stop reporting cache hits too.
The thread notes that router scoring likely includes aggregate caching rates but does not penalize individual reductions, allowing providers to quote lower prices to attract order flow while actually charging more by caching less.
Separately, a user ran a real six-position test on vLLM with 1,206 requests and zero errors, finding that one configuration change produced a 189.5% to 246.2% throughput jump, suggesting significant headroom exists for providers to manipulate performance metrics.
Inference routing has become a critical layer between application developers and frontier model providers. If caching and throughput metrics can be gamed, the entire price-discovery mechanism breaks down. Honest providers face a choice between losing traffic or gaming the system themselves, and users end up paying more for less performance.
Tarun who proven to be an incredible economic, finance, and DeFi game theory researcher, is now dedicating some of his time to the economics of AI — keep an eye on his AI research!