AGENTS · EDITORIAL CLASSIFICATION · NOT A BENCHMARK
DO I NEED THE BIG MODEL FOR…
Let an agent change a production codebase
Frontier-model jobYES — USE THE STRONG MODEL
“The model premium is small relative to the blast radius of a bad change.”
WHY THIS VERDICTSTART WITH
START WITH
THE WORK.
- 01
The task requires several dependent reasoning steps.
- 02
Several plausible interpretations can remain live.
- 03
A plausible wrong answer could create material cost or harm.
RECOMMENDED SETUPStrongest suitable model + tools + task-specific evals
NON-NEGOTIABLE
Sandbox execution, require tests and review, and scope credentials tightly.
REASONING5/5
AMBIGUITY4/5
ERROR COST5/5
CONTEXT5/5