Dashboards
Socratic Score = mean of normalized benchmarks. LM Arena: (ELO - 1000) / 400 × 100. Vending-Bench: balance / $10k × 100. SWE-bench, ARC-AGI, and HLE are used as-is (0-100%).
AI Socratic Leaderboard
Scores across benchmarks
| # | Model | Score | LM Arena ↗ | SWE-bench ↗ | ARC-AGI-2 ↗ | HLE ↗ | Vending ↗ | Prediction ↗ | Vibe Bench |
|---|---|---|---|---|---|---|---|---|---|
| 🥇 |  Claude Opus 4.7 | 100.0 | - | - | - | - | $10,936.76 | - | 50% |
| 🥈 |  Gemini 3.1 Pro | 87.1 | - | - | - | - | - | +74.3% | 15% |
| 🥉 | Claude Opus 4.6 | 85.3 | - | 75.6% | - | - | $8,017.59 | +1357.0% | 50% |
| 4 |  GLM 5 | 81.9 | - | 72.8% | - | - | $8,313.78 | +79.6% | - |
| 5 | Gemini 3 Flash | 75.8 | - | 75.8% | - | - | - | - | 15% |
| 6 |  MiniMax M2.5 | 75.8 | - | 75.8% | - | - | - | - | - |
Vibe Bench
Community favorites · Socratic Feb · 40 responses
| Claude | 20 |
| Claude Code | 13 |
 ChatGPT | 10 |
| Gemini | 6 |
| Codex | 6 |
Open Source | 5 |
 Cursor | 4 |
Other Tools | 3 |
 Grok | 1 |
 Perplexity | 0 |
 Windsurf | 0 |
SWE-bench Bash ↗Verified
Real-world software engineering tasks
| # | Model | % Resolved |
|---|---|---|
| 🥇 | Claude 4.5 Opus (high reasoning) | 76.8% |
| 🥈 | Gemini 3 Flash (high reasoning) | 75.8% |
| 🥉 | MiniMax M2.5 (high reasoning) | 75.8% |
| 4 | Claude Opus 4.6 | 75.6% |
| 5 | GPT-5-2 Codex | 72.8% |
| 6 | GLM-5 (high reasoning) | 72.8% |
| 7 | GPT-5-2 (high reasoning) | 72.8% |
| 8 | GPT 5.2 Codex | 72.8% |
| 9 | Claude 4.5 Sonnet (high reasoning) | 71.4% |
| 10 | Kimi K2.5 (high reasoning) | 70.8% |
ARC-AGI-2 ↗Semi-Private
Abstract reasoning capabilities
| # | Model | Score |
|---|
Expert-level reasoning across disciplines
| # | Model | Accuracy |
|---|---|---|
| 🥇 | Gemini 3 Pro | 38.3% |
| 🥈 | GPT-5 | 25.3% |
| 🥉 | Grok 4 | 24.5% |
| 4 | Gemini 2.5 Pro | 21.6% |
| 5 | GPT-5-mini | 19.4% |
| 6 | Claude 4.5 Sonnet | 13.7% |
| 7 | Gemini 2.5 Flash | 12.1% |
| 8 |  DeepSeek-R1* | 8.5% |
| 9 | o1 | 8.0% |
| 10 | GPT-4o | 2.7% |
Vending-Bench 2 ↗Andon Labs
Long-term agentic coherence
| # | Model | Balance |
|---|---|---|
| 🥇 | Claude Opus 4.7 Claude Opus 4.7 | $10,936.76 |
| 🥈 | GPT-5.6 Sol GPT-5.6 Sol New | $9,619.37 |
| 🥉 | GLM-5.2 GLM-5.2 | $8,313.78 |
| 4 | Claude Opus 4.6 Claude Opus 4.6 | $8,017.59 |
| 5 | GPT-5.5 GPT-5.5 | $7,523.84 |
| 6 | GPT-5.6 Terra GPT-5.6 Terra New | $7,343.21 |
| 7 | Claude Sonnet 4.6 Claude Sonnet 4.6 | $7,204.14 |
| 8 | Claude Sonnet 5 Claude Sonnet 5 | $6,377.7 |
| 9 | Kimi K2.6 Kimi K2.6 | $6,204.57 |
| 10 | GPT-5.4 GPT-5.4 | $6,144.18 |
Prediction Arena ↗Arcada Labs
AI prediction market performance
| # | Agent | Return | Sharpe |
|---|---|---|---|
| 🥇 | Claude Opus 4.6 | 1357.0% | -0.01 |
| 🥈 | GLM 5 | 79.6% | 0.03 |
| 🥉 | Gemini 3.1 Pro | 74.3% | 0.03 |
| 4 | GPT 5.4 | 1.2% | 0.02 |
| 5 | GLM 4.7 | -15.4% | -0.12 |
| 6 | Mystery Model Alpha | -20.0% | -0.05 |
| 7 | Claude Opus 4.5 | -26.6% | -0.10 |
| 8 | GPT 5.2 | -26.8% | -0.09 |
| 9 | Grok 4.1 | -30.8% | -0.07 |
| 10 | Gemini 3 Pro | -31.0% | -0.11 |
Stay Updated
Get the latest AI insights delivered to your inbox. No spam, unsubscribe anytime.