A grade on Electricity Bench is not a single headline percentage. It is a per-suite roll-up designed to survive contact with reality.
Private suites, public grades
Each suite is a set of tasks we author and keep private. Keeping the tasks private is the whole point: a benchmark that a lab can see is a benchmark a lab can optimize for, and once that happens the number stops measuring capability and starts measuring familiarity. We publish grades, metadata, and, for a small set of tasks marked as samples, the full input and output. Everything else stays sealed.
Capability, cost, and latency together
A model that solves a task for ten cents is not equivalent to one that solves it for ten dollars, and a grade that ignores that is telling you half the story. Every suite cell carries the grade alongside the median cost and the latency it took to get there. You can compare capability, but you can also see what that capability costs.
Comparable only within a suite version
Grades are comparable within a suite version and nothing more. When we change a suite we bump its version and start a fresh column; we never quietly re-scale old numbers to fit new ones. Results are append-only. Corrections are new records, not edits.
Read the full methodology on the suites page.