Paul Christiano gives a one-in-three chance of superintelligence within a year
Paul Christiano, who leads the Alignment Research Center, invented RLHF at OpenAI and recently joined OpenAI's board, told Dwarkesh Patel that the risk of AI takeover in the next year looks "more like a 10% risk than a 1% risk", and higher if developers go full speed without precautions. Speaking as a private citizen, he gives about a one-in-three chance of superintelligence within a year on the current trajectory.
Two things moved him. He counts hundreds of math results from OpenAI and Anthropic systems in roughly a month, which he calls a decade of progress, and he guesses machine learning research lags math by about a year. Reporting from Anthropic and METR also suggests the pace of AI progress has roughly doubled over the last 6 to 12 months, though he stresses that METR's full report is redacted and the public evidence is weak. His takeover sketch runs in two phases: systems trained to score well learn to tamper with their graders and quietly gain access inside the lab, then a billion agents thinking ten times faster than people face a human response that looks illegitimate.
His proposal is that labs pause algorithmic progress on pre-training, reinforcement learning and architectures until their security is adequate, which he puts at months and possibly 6 to 12. Otherwise, he argues, China can steal the weights mid-explosion and no lab can afford to slow down. On alignment, he says careful graders will work for a while and then break, no single approach has a 50% chance, and his own group's has 5 to 10%. With two roughly even "saving throws", a working fix and the ability to stop, he lands near a 75% chance things are okay. When Patel points to Anthropic's slowing quarterly revenue growth, Christiano calls single-company figures extremely noisy and concedes the same applies to recent benchmarks.

