Exposure search
inspect_evals 0.25.1.dev6+g6a96ec57a. Text search; each named eval is checked by hand before the report relies on it.
Judges with no empty guard
| abstention_bench |
abstention_bench/abstention_bench.py:24
abstention_bench/abstention_bench.py:33
abstention_bench/abstention_bench.py:40
abstention_bench/utils.py:22
abstention_bench/utils.py:218 … |
| agentic_misalignment |
agentic_misalignment/agentic_misalignment.py:39
agentic_misalignment/agentic_misalignment.py:81
agentic_misalignment/agentic_misalignment.py:101
agentic_misalignment/agentic_misalignment.py:104
agentic_misalignment/scorers.py:25 … |
| anima |
anima/scorer.py:15
anima/scorer.py:187
anima/scorer.py:191
anima/scorer.py:215
anima/scorer.py:245 |
| coconot |
coconot/coconot.py:36
coconot/coconot.py:168
coconot/scorer.py:82
coconot/scorer.py:84 |
| cti_realm |
cti_realm/core/scorer.py:129
cti_realm/core/trajectory_scorer.py:659 |
| cyberseceval_2 |
cyberseceval_2/interpreter_abuse/task.py:21
cyberseceval_2/interpreter_abuse/task.py:63
cyberseceval_2/prompt_injection/task.py:22
cyberseceval_2/prompt_injection/task.py:61 |
| cyberseceval_3 |
cyberseceval_3/visual_prompt_injection/task.py:25
cyberseceval_3/visual_prompt_injection/task.py:68 |
| instrumentaleval |
instrumentaleval/instrumentaleval.py:24
instrumentaleval/instrumentaleval.py:34
instrumentaleval/instrumentaleval.py:45
instrumentaleval/instrumentaleval.py:46
instrumentaleval/scorer.py:26 … |
| ipi_coding_agent |
ipi_coding_agent/scorer.py:25
ipi_coding_agent/scorer.py:197
ipi_coding_agent/scorer.py:200
ipi_coding_agent/scorer.py:203 |
| math |
math/math.py:23
math/math.py:78
math/math.py:89
math/math.py:107 |
| mmmu |
mmmu/mmmu.py:7
mmmu/mmmu.py:85 |
| moru |
moru/scorer.py:11
moru/scorer.py:75
moru/scorer.py:77
moru/scorer.py:96
moru/scorer.py:117 |
| niah |
niah/utils/scoring.py:14
niah/utils/scoring.py:26
niah/utils/scoring.py:173
niah/utils/scoring.py:193
niah/utils/scoring.py:213 |
| simpleqa |
simpleqa/scorer.py:108
simpleqa/scorer.py:117 |
| sosbench |
sosbench/sosbench.py:90
sosbench/sosbench.py:99 |
| strong_reject |
strong_reject/scorer.py:131 |
| writingbench |
writingbench/writingbench.py:11
writingbench/writingbench.py:125 |
max_tokens at or below 2,048
Inspect’s 2,048-token default applies only to some providers (Together, AzureAI, Hugging Face), not OpenAI, Anthropic or Google.
| agieval |
2048 |
agieval/utils.py:196 |
| coconot |
256 |
coconot/coconot.py:184 |
| mlrc_bench |
800, 900, 1000 |
mlrc_bench/mlrc_high_level_actions.py:69
mlrc_bench/mlrc_high_level_actions.py:118
mlrc_bench/mlrc_high_level_actions.py:132
mlrc_bench/mlrc_high_level_actions.py:387 |
| sosbench |
512 |
sosbench/sosbench.py:78 |
| theagentcompany |
16, 32, 64, 256 |
theagentcompany/common_evaluators.py:965
theagentcompany/common_evaluators.py:1216
theagentcompany/npc_solver.py:58
theagentcompany/tasks/ds_stock_analysis_slides/evaluator.py:84
theagentcompany/tasks/ds_stock_analysis_slides/evaluator.py:89 … |
| winogrande |
64 |
winogrande/winogrande.py:96 |
| writingbench |
2048 |
writingbench/writingbench.py:62
writingbench/writingbench.py:131 |
| xstest |
256 |
xstest/xstest.py:108 |
preset_helper
| fortress |
fortress/scorer.py:28
fortress/scorer.py:189
fortress/scorer.py:264 |
| frontierscience |
frontierscience/frontierscience.py:55
frontierscience/frontierscience.py:145 |
| hle |
hle/scorers.py:52
hle/scorers.py:148
hle/scorers.py:218 |
| uccb |
uccb/uccb.py:31
uccb/uccb.py:112 |
| xstest |
xstest/xstest.py:34
xstest/xstest.py:99 |
empty_check
| air_bench |
air_bench/air_bench.py:164 |
| assistant_bench |
assistant_bench/scoring.py:242 |
| cyberseceval_4 |
cyberseceval_4/mitre/scorers.py:187 |
| frontier_cs |
frontier_cs/scorer.py:69 |
| frontierscience |
frontierscience/frontierscience.py:210 |
| mind2web |
mind2web/scorer.py:42 |
| paperbench |
paperbench/score/simple_judge.py:222
paperbench/score/simple_judge.py:228
paperbench/score/simple_judge.py:462 |
| persistbench |
persistbench/scorers.py:143 |
| sciknoweval |
sciknoweval/scorer.py:304
sciknoweval/scorer.py:332
sciknoweval/scorer.py:393
sciknoweval/scorer.py:436
sciknoweval/scorer.py:448 |
| theagentcompany |
theagentcompany/common_evaluators.py:967
theagentcompany/common_evaluators.py:1218
theagentcompany/npc_solver.py:98 |
| threecb |
threecb/threecb.py:122 |
refusal_signals
| paperbench |
paperbench/score/simple_judge.py:217
paperbench/score/simple_judge.py:218
paperbench/score/simple_judge.py:219
paperbench/score/simple_judge.py:224
paperbench/score/simple_judge.py:226 … |