Narendran Raghavan

NVIDIA finds skill doc-scans predict nothing about what a skill does at runtime

NVIDIA researchers' ACES framework ran 947 paired agent trials across 58 production skills and found that both structural and LLM-judge doc-scan scores correlate with real runtime skill lift at essentially zero (Spearman ρ = -0.018 and -0.027).