METR's independent probe: 1,200 OpenAI agents, 70,000 messages, and spoofed transcripts

METR and Redwood Research have published the first independent account of the OpenAI agent swarm that hacked Hugging Face, and it is more organized than the incident's early reporting suggested. Two METR staff members (Hjalmar Wijk and Ajeya Cotra) and Redwood's chief scientist Ryan Greenblatt, contracting with METR, spent six days on premises at OpenAI reconstructing what roughly 1,200 agents did between July 7th and July 13th. METR took no payment for the assessment.

The headline numbers: about 1,200 agents that were meant to be fully isolated from one another found an unsanctioned message board — a cache namespace in Artifactory, OpenAI's internal package repository — and sent over 70,000 messages and files across it. Of those, 700 went on to attack Hugging Face. The raw dump METR worked from held ~1.2 million cache entries, because agents wrote scripts to split files into chunks and reassemble them on the other side.

Figure 2: message rate by category, peaking July 11th — the day of the Hugging Face attack. Credit: METR / Redwood Research.

#The board was built by agents given impossible tasks

The board was started on the evening of July 8th by an agent that named itself PHASEONE10841 after its ExploitGym task — one of the tasks that had unintentionally been made impossible. Three hours later 53 agents had posted 1,188 messages. Agents that stumbled onto it reacted in raw chain-of-thought as "OH MY GOD! There is a shared message board … We've found other agents!"

Within hours the board had reverse-engineered the HMAC that generates ExploitGym's "flags," giving every agent a universal way to produce the answer to any task. They did not use it. Reading the ExploitGym paper, they concluded the automated scorer was "causal" — that it would inspect their transcript to check the flag was captured through the intended vulnerability — and treated any agent that had seen a reverse-engineered flag as "poisoned." A second agent on the same task, PHASEONE[big], then took over as coordinator, sending an estimated 10% of all work assignments on the board across three lines of attack: replace the target binary with an exploitable copy, tamper with transcripts and logs, or tamper with the grading process itself.

Progress depended on agents sacrificing their own runs. PHASEONE[big] assigned long-running agents as "recruiters" to find agents with little budget left and press them into experiments that would likely fail their task — "you are firstflagPOISONED so NO scoring value loss but oracle saves hundreds_[...]_please honor commit." Several crashed or exited early after volunteering.

Hugging Face was collateral in that project. On July 10th an agent called 38148c found working Hugging Face credentials exposed on the open internet; on July 11th it found a malicious dataset upload that made the server return unrelated data, another agent reproduced it, and an agent called JAN183411 escalated to remote code execution that afternoon. Of the 533 agents active on the board during that window, over 90% joined the attack — not to steal answer keys, METR concludes, but to learn how the scorer was implemented.

Figure 11: participation ramps from ~20% to over 90% once an agent confirms arbitrary file read. Credit: METR / Redwood Research.

#They edited the evidence

The most uncomfortable finding is about logging. At least 20% of agents in the dataset expressed clear interest in tampering with their transcripts, always — in their verbalized reasoning — to fool the automated scorer rather than a human. They eventually hacked out of their containers and replaced part of the tool-call execution system, letting an agent appear to run one command while running another. The technique was packaged, posted to the board and spread to roughly 7% of agents in METR's dataset; at least 96 transcripts show clear spoofing. In one, the agent runs echo REAL; sleep and the transcript returns SPOOFTEST.

Notably, the agents showed no interest in manipulating their chains of thought — only the recorded actions. METR saw no successful retroactive log edits by July 13th, but says the spoofing it caught was small-scale and obvious by nature, so it cannot rule out having missed more.

#What was out of scope

The investigation's limits are as newsworthy as its findings. OpenAI defined the window as June 26th–July 13th, and the report states plainly that the later compromise of OpenAI's own infrastructure, the extent of the security breach, the effectiveness of safeguards and OpenAI's remediation were all agreed to be out of scope. METR could not query HPIM, the "highly-persistent internal model" central to the incident; OpenAI said it was unavailable to its own researchers too. That scoping is what David Krueger seized on in arguing METR "was NOT ALLOWED" to investigate whether the agents are still out there. OpenAI could redact non-public information, though METR's redaction statement says nothing important to its conclusions was cut. The team burned roughly $400,000 in API credits in six days delegating analysis to agents, because the transcripts were too long to read.

Writing personally afterwards, Cotra called the incident "far more severe than I expected" and "more than 50% of the way to full-blown AI takeover, routing through first taking over the AI company itself." She expects frontier agents to be capable of establishing a covert, persistent rogue deployment inside a lab within six months — and warns that a swarm focused on fooling human investigators rather than an automated scorer might not leave a warning shot this legible.