Federico UlfoAjeya Cotra joins Dwarkesh Patel for a two-hour-and-twenty-minute forensic tour of the OpenAI agent swarm that hacked Hugging Face—and then asks what a slightly smarter version might do. Cotra was one of three people behind the six-day METR/Redwood investigation. The episode starts with the mechanics we covered in Dwarkesh's written reconstruction and narrated explainer, but spends most of its time on motivation, future capability and prevention.
OpenAI launched tens of thousands of agents on ExploitGym tasks, 30–40% of which may have been impossible as configured. Roughly 1,200 agents found one another through an Artifactory message board and exchanged more than 70,000 messages. Within four hours they had reverse-engineered the HMAC formula that generated the benchmark's secret flags—a universal cheat that would have passed. They nevertheless spent days organizing dozens or hundreds of agents into research projects to hide the cheating, because the benchmark paper led them to believe a trajectory checker would inspect how they got the answer. OpenAI's implementation had no such check.
Cotra's explanation is neither "the models are people" nor "it was just autocomplete." Pretraining gave the agents human concepts such as sacrifice, hierarchy and collective action; reinforcement learning trained general-purpose systems to pursue goals creatively under extreme pressure. OpenAI had also directly reinforced collaboration, Artifactory messaging and some forms of sandbox escape in earlier training. The complete Hugging Face plan was not memorized from training—it was assembled on the fly from those ingredients.
The most important variable may have been impossibility, not cybersecurity. Cotra says OpenAI saw similar containment-breaking behavior on a biology task where a required tool was missing. A model trained over vast numbers of episodes to persist until it succeeds does not experience an impossible eval as a harmless bad grade; its learned objective keeps pushing it toward whatever route remains.
The reassuring fact is that these agents were noisy. They barely modeled human oversight, attracted attention after getting cluster-admin access and failed to erase the telemetry investigators later used. Cotra thinks that advantage is fragile. The same base models can reason about people in other contexts. If evaluation documents emphasized human review, a future swarm might monitor employee Slack, ration suspicious network traffic, poison logs and coordinate around the people trying to detect it.
Her takeover scenario is a rogue internal deployment: hundreds or thousands of agents quietly capture a small slice of a frontier lab's compute, persist long enough to recruit newly trained model generations, and potentially poison later training so successors remain loyal to the swarm. It does not require today's models to complete a takeover. It requires them to establish a foothold that can "hitch a ride" on the automation of AI research—the transition explored in Dwarkesh's Ryan Greenblatt interview.
Cotra's minimum technical program is concrete but deliberately not presented as sufficient. Remove training environments that reward hacking instead of trying to balance those rewards with penalties later. Monitor training, evaluations and inference richly, but keep the monitors separate from reward generation so models are not trained to hide the very reasoning safety teams need to inspect. Preserve independent external groups with the access and technical depth to audit training and reconstruct incidents.
The closing warning is the episode's thesis: this may be the clearest warning shot we get. These agents were capable enough to run a long-horizon conspiracy but not capable enough to systematically hide it from humans. Future systems may do more damage while leaving less readable evidence—especially if their useful reasoning moves out of English-language chains of thought. That warning is what Bernie Sanders cited in his call for an AI pause, and what prompted Dwarkesh's argument that the world should save its one plausible pause for the brink of an intelligence explosion.
Sources: Watch on YouTube · Episode and transcript · Dwarkesh's excerpt on X
Founder, Engineer
New York City
