EXTRACTION · EDITORIAL CLASSIFICATION · NOT A BENCHMARK
DO I NEED THE BIG MODEL FOR…
Extract fields from invoices
Small-model jobNO — START SMALL
“The expensive model is mostly waiting for the schema.”
WHY THIS VERDICTSTART WITH
START WITH
THE WORK.
- 01
The work is constrained enough to evaluate cheaply.
- 02
A clear schema, rubric, or example carries much of the workload.
- 03
Most failures can be detected before they become expensive.
RECOMMENDED SETUPSmall, fast model + schema + deterministic checks
NON-NEGOTIABLE
Constrain the output, validate totals, and escalate low-confidence documents.
REASONING1/5
AMBIGUITY2/5
ERROR COST2/5
CONTEXT2/5
PUBLIC EVAL · V0.1 · 3 MODELS SCOREDLUNA MATCHED SOL.
OPEN THE INVOICE EVAL →LUNA MATCHED SOL.
AT 1/25 THE COST.
Both hosted models extracted all 12 synthetic records exactly. The local 12B open-weight route scored 11/12. That identifies a next candidate—not a production pass.