What does the next training paradigm look like?

Dwarkesh describing a question

Here's a question I find confusing and interesting and which actually tells us a lot about the nature of current AI progress:

Why has progress on computer use been so slow? Computer use is so clearly verifiable.

I think the answer is that it is not enough for a domain to be verifiable.

It also has to be very grindable—in the sense that you can run lots of parallel rollouts against a deterministic and replayable simulator.

If you're trying to make a model better at coding, you can create an environment that has a software repo with some missing feature that you've tasked the AIs with creating, and then you have a thousand parallel agents just go at the problem, each with their identical copy of the container.

But this doesn't work with computer use—at least not trivially. You can't have a thousand agents go try the same checkout flow on Amazon. Because Andy Jassy will find and detect your bots and shut your ass down.

How would we train an AI to build a business? How would you make an AI that's really good at winning court cases? Or having a profitable day trading in the markets? Or helping a candidate win an election?

What is the RL environment to make an AI as good at politics as Lyndon Johnson, or as good at building a space launch business as Elon Musk?

The rollout requires interacting with the world and cannot be recreated simply within the datacenter. And the outer loop verification may take months or years of real world actions to elicit, and cannot be re-observed by perturbing the model's actions thousands of times in parallel so that you can isolate what exactly the model did that actually worked.

#The gist

The frontier labs are making one big bet: train AIs on millions of verifiable tasks across thousands of RL environments and you'll get a general problem-solving agent — one that can grind through open-ended work for weeks despite errors and ambiguity. Optimists argue the current paradigm's known deficits (extreme data inefficiency, no continual learning) get steamrolled by scale, the same way classic NLP problems collapsed under enough compute.

But verifiability isn't the whole story. A domain also has to be grindable — cheap to run thousands of parallel rollouts against a deterministic, replayable simulator. That's why coding races ahead while computer use lags: you can spin up a thousand identical containers for a repo, but you can't point a thousand bots at Amazon's checkout. Many of the skills we actually care about — building a business, winning court cases, running a campaign — can't be simulated this way at all. Their outer-loop verification takes months or years in the real world and can't be replayed.

So the deeper bottleneck is sample efficiency. Because so much real-world knowledge is sparse and idiosyncratic, AIs need to learn from scarce, unstructured, ambiguous signal. And whatever they do learn in-context today is mostly stranded: 30–50% of a lab's compute goes to inference, the place where the most valuable signal (what orgs use the model for, where it fails) actually shows up — and none of it makes it back into the weights. It's a brilliant grad student who's never allowed to take an internship.

Fixing this means continual learning — consolidating insight back into the weights rather than hoarding an ever-bigger context. But gradient updates are sample-inefficient, which is why shipped online-learning systems only learn one shared objective across millions of users (e.g. Cursor's Tab model). Dwarkesh floats two more promising directions: on-policy self-distillation (teach the base model to match a context-rich "veteran" version of itself — denser signal than RL, more targeted than SFT), and the speculative dreaming — models building their own simulators to rehearse against, a potential fourth scaling axis after pretraining, RL, and inference.

His 2027–28 sketch: RLVR produces an agent competent enough to be deployed broadly; effective context stretches to a week of real co-working; a thumbs-up at the end triggers a distillation pass (OPSD, dreaming, or something new) that folds the week's learnings into the weights. Capabilities then expand outward from the verifiable domains, round by round — and AIs improve mostly from being deployed in the economy rather than from pre-release training. Exciting, and a little scary.

Narrated from Dwarkesh Patel's essay "The Next Paradigm." Full piece and footnotes at dwarkesh.com.