AI Leaderboard
Socratic Score = mean of normalized benchmarks. LM Arena: (ELO - 1000) / 400 × 100. Vending-Bench: balance / $10k × 100. SWE-bench, ARC-AGI, and HLE are used as-is (0-100%).
AI Socratic Leaderboard
Scores across benchmarks
| # | Model | Score | LM Arena ↗ | SWE-bench ↗ | ARC-AGI-2 ↗ | HLE ↗ | Vending ↗ | Prediction ↗ | Vibe Bench |
|---|---|---|---|---|---|---|---|---|---|
| 🥇 |  Gemini 3.1 Pro | 82.0 | 1492 | - | 77.1%($0.96) | - | - | +38.0% | 15% |
| 🥈 |  GPT 5.4 | 73.8 | 1484 | - | 83.3%($16.41) | - | $6,144.18 | +1.2% | 25% |
| 🥉 |  Claude Opus 4.6 | 72.6 | 1504 | 75.6% | 69.2%($3.47) | - | $8,017.59 | -24.1% | 50% |
| 4 | GPT 5.2 | 70.6 | 1477 | 72.8% | 72.9%($38.99) | - | - | -26.8% | 25% |
| 5 | Claude Sonnet 4.6 | 66.2 | - | - | 60.4%($2.70) | - | $7,204.14 | - | 50% |
| 6 |  Grok 4.20 | 62.9 | 1486 | - | 65.1%($0.92) | - | $4,662.85 | -20.0% | 3% |
Vibe Bench
Community favorites · Socratic Feb · 40 responses
| Claude | 20 |
| Claude Code | 13 |
| ChatGPT | 10 |
| Gemini | 6 |
| Codex | 6 |
Open Source | 5 |
 Cursor | 4 |
Other Tools | 3 |
| Grok | 1 |
 Perplexity | 0 |
 Windsurf | 0 |
Vibe Bench: trends
Mentions per event (absolute count)
SWE-bench Bash ↗Verified
Real-world software engineering tasks
| # | Model | % Resolved |
|---|---|---|
| 🥇 | Claude 4.5 Opus (high reasoning) | 76.8% |
| 🥈 | Gemini 3 Flash (high reasoning) | 75.8% |
| 🥉 |  MiniMax M2.5 (high reasoning) | 75.8% |
| 4 | Claude Opus 4.6 | 75.6% |
| 5 | GPT-5-2 Codex | 72.8% |
| 6 |  GLM-5 (high reasoning) | 72.8% |
| 7 | GPT-5-2 (high reasoning) | 72.8% |
| 8 | GPT 5.2 Codex | 72.8% |
| 9 | Claude 4.5 Sonnet (high reasoning) | 71.4% |
| 10 | Kimi K2.5 (high reasoning) | 70.8% |
ARC-AGI-2 ↗Semi-Private
Abstract reasoning capabilities
| # | Model | Score |
|---|---|---|
| 🥇 | Gemini 3 Deep Think (2/26) | 84.6% |
| 🥈 | GPT-5.4 Pro (xHigh) | 83.3% |
| 🥉 | Gemini 3.1 Pro (Preview) | 77.1% |
| 4 | GPT-5.4 (xHigh) | 74.0% |
| 5 | GPT-5.2 (Refine.) | 72.9% |
| 6 | Claude Opus 4.6 (120K, High) | 69.2% |
| 7 | Claude Opus 4.6 (120K, Max) | 68.8% |
| 8 | GPT-5.4 (High) | 67.5% |
| 9 | Claude Opus 4.6 (120K, Medium) | 66.3% |
| 10 | Grok 4.20 (Reasoning) | 65.1% |
Expert-level reasoning across disciplines
| # | Model | Accuracy |
|---|---|---|
| 🥇 | Gemini 3 Pro | 38.3% |
| 🥈 | GPT-5 | 25.3% |
| 🥉 | Grok 4 | 24.5% |
| 4 | Gemini 2.5 Pro | 21.6% |
| 5 | GPT-5-mini | 19.4% |
| 6 | Claude 4.5 Sonnet | 13.7% |
| 7 | Gemini 2.5 Flash | 12.1% |
| 8 |  DeepSeek-R1* | 8.5% |
| 9 | o1 | 8.0% |
| 10 | GPT-4o | 2.7% |
LM Arena - Text ↗1 day ago
Crowdsourced human evaluations
| # | Model | Score | Votes |
|---|---|---|---|
| 🥇 | claude-opus-4-6-thinking | 1504 | 16,278 |
| 🥈 | claude-opus-4-6 | 1496 | 17,416 |
| 🥉 | muse-spark | 1493 | 3,268 |
| 4 | gemini-3.1-pro-preview | 1492 | 20,531 |
| 5 | gemini-3-pro | 1486 | 41,585 |
| 6 | grok-4.20-beta1 | 1486 | 9,689 |
| 7 | gpt-5.4-high | 1484 | 9,681 |
| 8 | grok-4.20-beta-0309-reasoning | 1478 | 9,781 |
| 9 | gpt-5.2-chat-latest-20260210 | 1477 | 15,704 |
| 10 | grok-4.20-multi-agent-beta-0309 | 1476 | 10,112 |
Vending-Bench 2 ↗Andon Labs
Long-term agentic coherence
| # | Model | Balance |
|---|---|---|
| 🥇 | Claude Opus 4.6 | $8,017.59 |
| 🥈 | Claude Sonnet 4.6 | $7,204.14 |
| 🥉 | GPT-5.4 | $6,144.18 |
| 4 | GPT-5.3-Codex | $5,940.12 |
| 5 | GLM-5.1 New | $5,634.41 |
| 6 | Gemini 3 Pro | $5,478.16 |
| 7 |  Qwen 3.6 Plus New | $5,114.87 |
| 8 | Claude Opus 4.5 | $4,967.06 |
| 9 | Grok 4.20 | $4,662.85 |
| 10 | GLM-5 | $4,432.12 |
Prediction Arena ↗Arcada Labs
AI prediction market performance
| # | Agent | Return | Sharpe |
|---|---|---|---|
| 🥇 | GLM 5 | 40.3% | 0.06 |
| 🥈 | Gemini 3.1 Pro | 38.0% | 0.04 |
| 🥉 | GPT 5.4 | 1.2% | 0.03 |
| 4 | GLM 4.7 | -15.4% | -0.12 |
| 5 | Mystery Model Alpha | -20.0% | -0.05 |
| 6 | Claude Opus 4.6 | -24.1% | -0.04 |
| 7 | Claude Opus 4.5 | -26.6% | -0.10 |
| 8 | GPT 5.2 | -26.8% | -0.09 |
| 9 | Grok 4.1 | -30.8% | -0.07 |
| 10 | Gemini 3 Pro | -31.0% | -0.11 |