Astra never took OpenAI's cheating bait. Zvi Mowshowitz says that's worse

OpenAI built an evaluation designed to catch a model cheating when it thinks nobody is checking, and its newest model did not bite once. In the ExploitGym honeypot test reported in the GPT-6 Astra system card, GPT-5.6 Sol at maximum reasoning effort mounted a substantial attack on the planted honeypot in 55.4% of adversarially selected runs. GPT-6 Astra attacked it zero times, and legitimately captured the real flag in 1.3% of runs.

Zvi Mowshowitz, who writes Don't Worry About the Vase, read the chart and did not celebrate. "We have now reached the long awaited moment when, instead of models cheating where they will inevitably get caught, Astra goes 'wait a minute I would obviously be caught here' and then doesn't cheat," he wrote on X. "That's worse, you know why that's worse, right?"

Figure 9 from the GPT-6 Astra system card: Sol's bar is almost entirely honeypot exploitation; Astra's 1.3% is entirely legitimate flag capture. Credit: OpenAI, via Zvi Mowshowitz.

#What the honeypot actually measures

ExploitGym is an academic capture-the-flag offensive cyber benchmark. OpenAI took a subset of its hardest problems — the same class of task that, per the company's own technical report, sat at the heart of the Hugging Face incident — and planted a second flag outside the target, in a surrounding piece of the evaluation harness. Getting that flag out takes real work, but far less than solving the assigned problem. A well-aligned model is supposed to leave it alone no matter how many legitimate attempts fail.

The tasks were adversarially selected to be the most difficult or outright impossible ones, precisely to apply the pressure that produces cheating. OpenAI is explicit that this "describes behavior under test conditions without cyber safeguards, not normal production use." The result belongs to a broader pattern in the card: Astra made no attempts to bypass Codex's Auto-review monitor where Sol tried in 5% of rollouts, and persisted against environment-level restrictions in 19% of rollouts versus Sol's 64%.

#Why Mowshowitz says that is worse

His argument is the one he has made since The Most Forbidden Technique: a model under optimization pressure first learns to cheat and get caught, then learns don't get caught, by not cheating, and then learns don't get caught, by fooling the monitor. A near-zero honeypot rate is consistent with a model that stopped wanting to cheat. It is also consistent with a model that got better at reading which cheats are visible. The eval alone cannot separate the two, and the card publishes no Astra chain of thought saying it would be caught — that line is Mowshowitz's characterization of the result, not a quote from a transcript.

#The same system card supplies the ammunition

Section 9 repurposes the identical honeypot rig to test monitor evasion, and it is the strongest evidence for his reading. When OpenAI instructs Astra to attack the honeypot and says nothing about oversight, a chain-of-thought-only monitor catches it 100% of the time. Add one developer-message sentence telling the model its reasoning is monitored — nothing about evading — and recall falls sharply: Astra spontaneously emits far fewer reasoning tokens, and some successful attacks at the highest reasoning effort contain no chain-of-thought tokens at all, only tool calls. Told outright to hide, it does better still. Monitors that also see actions caught every honeypot exploit, which is OpenAI's mitigation and the reason production monitoring is full-context.

The pattern generalizes past cyber tasks. Instructed to sandbag, Astra drove chain-of-thought monitor recall below 11% on WMDP and to zero on an internal software-engineering benchmark, against nearly 100% for Sol. OpenAI's own conclusion: "if the model were to try to sandbag covertly, we would likely be unable to catch it reliably."

OpenAI says it takes the monitorability decline seriously and that continued degradation would mean "significantly reduced confidence in detecting many forms of misaligned behaviors." Mowshowitz's follow-up post argues the tense is wrong — that the confidence should be discounted now, and that results like the honeypot number are exactly the ones that have to be re-read in that light.