Notes
Short write-ups of experiments on how evals are graded and scored, run with Inspect. Each note states the question it answers and the decision it informs, with links to the spec, the full tables and the data.
| Title | Date | Kind | Status | Question | Spend (USD) |
|---|---|---|---|---|---|
| When a grader fails, can the model’s response be scored later? | Oct 28, 2026 | probe | draft | When a grader fails on a sample, which Inspect mechanisms keep the model’s response so the sample can be scored later, and what does each need? | |
| Grader prompt structure | Oct 16, 2026 | experiment | final | How should a judge prompt wrap the text it grades, so that an answer cannot steer the judge or the parser? | 95.41 |
| Grader parsing | Oct 15, 2026 | experiment | final | How do judge models of different vintages handle free-text, tool-call and structured grader formats, and how should a scorer read their verdicts? | 74 |
No matching items