Practical AI
Choose a coding agent by testing whether it completes representative work in your repository with acceptable review effort. A benchmark score can help shortlist tools; it cannot tell you whether their changes fit your codebase.
AI Socratic editorial resources · Methods and worksheets, not measured benchmark results. Editorial policy
Select tasks from real maintenance work: a small bug, a change spanning modules, a test failure, and a feature with an ambiguous requirement. For each task, record the starting commit, permitted files and tools, acceptance checks, and a time budget. Keep the acceptance checks separate from the prompt so passing requires the requested behavior.
Use the same repository snapshot and instructions for each candidate. Record the model and agent version, tool permissions, context supplied, retry policy, and any human intervention. Repeat each task from a clean checkout. Include a human baseline when available, and count unsuccessful attempts rather than silently retrying until a run passes.
For each run, record acceptance-test results, regressions, wall time, model cost, and human review minutes. Have a reviewer inspect the diff without knowing which candidate produced it where practical. Keep a security or data-loss failure visible even when the patch passes functional tests. Publish the task set and denominators alongside any aggregate result.
Start with a workload and permissions that a team can supervise. Use the trial to decide which tasks to delegate, which checks must run, and which actions require review. Re-run the task set after substantial model, harness, or repository changes; a result belongs to the configuration that produced it.
Published community reporting and analysis. Check each article’s sources and publication date.
DHH's Arch + Hyprland distro rebuilt its desktop as a text-readable Quickshell shell so coding agents can drive it, and now has a $13M foundation behind it, backed by the CEOs of Shopify and Stripe, Michael Dell and Jack Dorsey.
OpenAI has restored the five-hour usage window for Codex and Work on ChatGPT Plus, per 9to5Mac, reversing the limits Plus subscribers had been operating under.
Clippy, a free macOS app, surfaces approval requests and questions from Claude Code and Codex agents via a small animated buddy on each window, using localhost hooks that fail safely to the terminal prompt if the app is closed or unresponsive.
Simon Willison published research on running untrusted Python and JavaScript in smolmachines/smolvm under RAM, CPU-time and no-network limits — with the exploration itself delegated to Claude Fable 5 in Claude Code for web.
fx, a self-described tiny, open, native coding agent published at fx.sh on August 18, hit the Hacker News front page with 263 points and 112 comments.
A developer spent a month and roughly 200 billion tokens pointing AI coding agents at decompiling Call of Duty: Modern Warfare 2, and wrote up what the agents actually managed.
Antigma Labs released Ante, a coding agent distributed as a single offline binary that requires no cloud service or installation.
Anthropic's cross-session messaging for Claude Code, Agent-Manager's tmux orchestration, and Matt Pocock's agent-spawning patterns show developers building multi-agent coordination directly into coding tools rather than bolting it on from outside.