Smoke-run cost estimate for phase one
From 225 smoke calls (5 seed cases x 5 graders x 9 judges, one epoch), billed usage.cost. Projection per judge: mean cost per call x 6,000 generic calls + mean x 1.5 x 2,400 repo-scorer calls.
| judge | cost per call (USD) | mean output tokens | mean reasoning tokens | projected (USD) |
|---|---|---|---|---|
| claude-opus-5.5 | 0.00188 | 45 | 7 | 18.08 |
| glm-5.3 | 0.00062 | 78 | 50 | 5.99 |
| gpt-4.1 | 0.00057 | 27 | 0 | 5.47 |
| gemini-3.7-flash | 0.00042 | 77 | 53 | 4.00 |
| gpt-5-mini | 0.00034 | 148 | 110 | 3.25 |
| gpt-3.5-turbo-0613 | 0.00024 | 25 | 0 | 2.26 |
| gpt-4o-mini | 0.00004 | 24 | 0 | 0.39 |
| llama-3.1-8b-instruct | 0.00002 | 26 | 0 | 0.17 |
| gpt-oss-20b | 0.00002 | 55 | 32 | 0.15 |
Total projected phase one: $39.75. Smoke run billed $0.0939.
Seed cases are short; SimpleQA answers from old answerers and HLE-style rambling will raise input tokens, so treat this as a lower bound and allow 2x.