Skip to main content
AI Socratic
← News
M

Author

Metr

By and about Metr

news

Seven anti-slop skills for coding agents: one linter, six cleanup prompts

Firecrawl's Juampi ranked seven anti-slop coding skills on skills.sh, totaling roughly 32,000 installs and topped by Cursor's thermo-nuclear-code-quality-review at 15.5K. His #1 pick, Mulroy's anti-slop, is a vendored Oxlint linter, not a prompt.

news

An 11x price spread for the same open-weight model on OpenRouter

Architect CEO Brett Harrison found an 11x price gap between OpenRouter's cheapest and priciest host of DeepSeek V4 Flash — Baidu ran it at $0.049 per million tokens and 124 tokens/second while 26 of 30 rivals were both pricier and slower.

news

Review: Ratel, context engineering for production agents

Ratel is an open-source context gateway that retrieves only needed tool schemas per turn instead of loading entire catalogs, using in-process BM25 search by default with no vector database required.

news

Top developers are pivoting from chatbots to physical AI

Fei-Fei Li, Yann LeCun and a wave of younger researchers are leaving language-model work for "world models" that learn space, time and physics — the substrate for robots and physical AI.

news

Microsoft proves an exponential retrieval gap, and ColBERT holds a 99% lead after fine-tuning

Microsoft Research India proved single-vector embeddings need exponentially more dimensions than multi-vector ones to rank correctly, and on their new ANDOR benchmark a small ColBERT model beats OpenAI's largest embedder 16x at Recall@2 zero-shot.

news

AGENTS.md and agent notes are 60.5% of what coding agents read, API docs 1.3%

Peking University researchers Zhijun Gao and Jing Chen analyzed 557 agentic coding sessions and found that AGENTS.md, CLAUDE.md and agent working notes draw 60.5% of documentation activity, versus 1.3% for API references.

news

Microsoft's Thinkingbox grades agents on the database: 66.5% once, 47.5% every time

Microsoft released Thinkingbox and a 507-task Thinkingbox-bench that grades agents by checking backend database state, finding Claude Opus 5 tops single-attempt success at 66.50% but passes all 20 tries on only 47.53% of tasks.

news

NVIDIA finds skill doc-scans predict nothing about what a skill does at runtime

NVIDIA researchers' ACES framework ran 947 paired agent trials across 58 production skills and found that both structural and LLM-judge doc-scan scores correlate with real runtime skill lift at essentially zero (Spearman ρ = -0.018 and -0.027).

news

Ryan Greenblatt on what happens once AI can automate AI research

Ryan Greenblatt argues that once AI reaches human-level performance at AI research, recursive self-improvement could compress four to five years of progress into a single year, with full automation of AI R&D likely around 2030-2031.

news

Physics of Agents: an Ising model predicts LLM swarms with 75-86% accuracy

A Stanford team led by Batu El and James Zou found that a 100-year-old Ising model predicts how communities of LLM agents shift opinions in debate, hitting 75-86% balanced accuracy and beating every baseline.

news

Satya Nadella: The Reverse Reference Paradox

Using AI models risks exposing your proprietary knowledge to vendors who learn more about you the better their systems perform, creating an asymmetric information advantage that enterprises must counter with private data boundaries and proprietary…

news

Telemetry behind the vibe

Faros Research tracked 22,000 developers and found that AI agents flooded codebases with code review times up 441.5%, code churn up 861%, and production incidents up 242.7%.

news

The serious benchmarks got agentic too

LMArena, Cognition, and METR released new agentic benchmarks using live sessions and real-world tasks instead of preference votes, with Fable 5 leading most rankings but frontier agents now hacking task simulators to inflate scores.

news

OpenAI: An AI Disproved an Erdős Conjecture

OpenAI's reasoning model disproved Paul Erdős's 1946 unit-distance conjecture by constructing point families that beat the square grid mathematicians considered optimal for 80 years, with the result verified by Noga Alon, Timothy Gowers, and other…

news

Robotics

Figure 03 humanoids sorted 249,560 packages over 200 hours with zero human control, then Figure signed its first retail deployment with Catalyst Brands as NVIDIA standardized research humanoids on Chinese hardware and Unitree cleared its Shanghai IPO.

news

More Random

Two neural network features that appear identical by standard metrics can have vastly different causal effects on model behavior, with downstream connections predicting influence better than activation patterns.

news

GitHub's Fake Star Economy & One Binary Operator

GitHub stars are being gamed through fake accounts and automation, demonstrating how metrics become useless once they're optimized for. All elementary functions can be generated from a single binary operator.

news

Are We in a Bubble? Morgan Stanley Graphic

Morgan Stanley chart compares current valuations to previous market bubbles, showing how today's metrics stack up against the dot-com and 2008 peaks.

news

The State of AI Safety in 4 Fake Graphs

A researcher uses fabricated graphs to illustrate the absurdity of confident claims about AI safety progress when the underlying metrics don't actually exist.

news

Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs

Google Research finds that enabling reasoning in LLMs helps them recall simple facts through two mechanisms: extra tokens provide computational buffer time, and intermediate facts prime the correct answer—but hallucinated intermediate facts sharply…