Every new AI model, graded on private benchmarks
We run our own niche task suites against each notable model the day it drops, and publish the graded scorecard here. The tasks stay private so nobody can train against them; the numbers come out in the open.
Every new model’s scorecard, the day it drops.
Leaderboard
| Model | Overall | structured-extraction |
|---|---|---|
| Claude Sonnet 5 (CLI) | — | A $0.0000 |
Grades are within-suite-version comparable only. See methodology.
How it works
1
Private suitesNiche task suites - agentic coding, extraction, automation - kept behind the curtain so they can never leak into training data.
2
Graded runsEach new model runs every suite under the same harness: capability, cost, latency, and reliability, with repeats for stability.
3
Public scorecardsOnly the graded outcome is published. Third-party claims are labeled context, never blended into our grades.