Magic says it matched DeepSeek V4 Pro Base with 50x less compute

Magic, the coding-model startup that has raised hundreds of millions of dollars without shipping a model, published a pretraining research update claiming its recipe is more than 10x more compute-efficient than the leading open-weight base models. The headline number: it says it matched DeepSeek V4 Pro Base using roughly 50x fewer FLOPs — about half of GPT-3's pretraining compute, or ~$0.5M on GB200. It then scaled 10x further, for roughly $4M, and says that run "meaningfully outperformed all publicly available open base models on perplexity evals." By its own fitted scaling laws, a model that capable would cost over $100M under DeepSeek V4 Pro's recipe.

The framing is deliberate. "Frontier pretraining is said to be a big-lab-only game," the post opens. "We don't have 100k chips yet, so there's only one way: algorithmic efficiency." Magic describes itself as "likely the smallest team in the world training trillion parameter models."

#How the comparison was made

Base models — pretrained checkpoints with no RL, SFT or other post-training — are notoriously prompt-sensitive, so Magic skipped sampling-based benchmarks entirely. It measured bits-per-byte loss on held-out data, which normalizes away tokenizer differences, then fit scaling laws to project the compute needed to reach a given capability level. The held-out sets were its own codebase plus private repositories acquired from other startups, recent low-citation research papers, and private competition math problems. Decontamination removed vendored open-source code and any document matching a 96-character window of normalized text against the training corpus.

The baselines are open-weight base models from DeepSeek, Moonshot and NVIDIA. Magic evaluated log-probabilities in both vLLM and SGLang on GB200 and GB300, says it found issues with some backends along the way, and partnered with Fireworks to re-verify the baselines in Fireworks' own inference engine. Compute figures use the 6·N·D approximation:

The ~50x match therefore sits at the smaller end of Magic's own ladder; the $4M scale-up is the 1.63e24 run, still about 6x below DeepSeek V4 Pro's total. A separate figure fits scaling laws across 167 domains, including ones Magic deliberately deprioritizes — notable people, local news, sports.

#No single trick

Magic is explicit that there is no one breakthrough here. In late 2024 it trained a small dense model built for very long contexts, and "initial pretraining scale-ups kept blowing up in a wide variety of ways." The fix was infrastructure discipline first: smooth convergence, low-precision training with FP32-equivalent quality, correct hyperparameter scaling rules, and "most importantly: hunt the bugs." What followed was "the multiplicative result of tens of changes across model architecture, optimizer, training objective, and data curation." Two findings worth noting for anyone using NanoGPT speedruns as a proxy: many changes that help tiny models do not help big ones, and some features present in most LLMs can be deleted with no loss at scale. Each candidate is validated on three models spanning two orders of magnitude of compute, and kept only if the power-law fit says it helps at scale.

As a sanity check that pretraining loss translates, a short math RL run — 16k chain-of-thought budget, started directly from the base model with no SFT or distillation — put the e24 model past 90% pass@1 on AIME26 (100% pass@16) at roughly 0.2% of its pretraining compute budget, averaging just over 6k CoT tokens.

#Why it matters

Everything here is self-reported and unreplicated, and — as Trending Topics notes — comes from a company that has not yet released a model. Magic discloses no architecture, parameter count or training data. It can decontaminate evals for its own models but not for the open-weight baselines, though it argues any contamination there would favor the competitors. And the comparison is base-model held-out loss, not post-trained or agentic performance.

With those caveats, the claim is a real challenge to the "compute is destiny" story: if a small team can buy a frontier-grade base model for single-digit millions, the moat is the recipe, not the cluster. One incidental result deserves attention on its own — Magic says it was surprised to find Nemotron 3 outperforming DeepSeek V4 Pro across the board in base-model loss, consistently across domains and inference engines, suggesting NVIDIA's weak post-RL benchmark showing reflects post-training rather than a weak pretrain. Magic says it will now scale long-horizon RL, publish an updated AGI Readiness Policy covering deployment gates and RL-training safety requirements, and eventually "release the thing."