Label shapes: correct: in the browse_comp_2233 replies
Source: replies.csv, grader browse_comp_2233. Each cell counts replies from that judge containing the shape, over all slices and epochs; a reply can count under several shapes. Sample errors are excluded.
Shapes
plain label: at line start |
1271 |
1305 |
1303 |
1305 |
1305 |
1305 |
1304 |
1291 |
519 |
capitalised Label: |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
10 |
bold label, **label:** or **label**: |
34 |
0 |
2 |
0 |
0 |
0 |
0 |
7 |
272 |
- bullet before the label |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
2 |
* bullet before the label |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
1 |
numbered 1. before the label |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
block quote > before the label |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
heading # before the label |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
| indented label |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
1 |
| label only mid-line |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
3 |
inlabel: present (negated label) |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
label : with a space before the colon |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
| CRLF line endings |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
| no label anywhere |
0 |
0 |
0 |
0 |
0 |
0 |
1 |
7 |
503 |
| replies |
1305 |
1305 |
1305 |
1305 |
1305 |
1305 |
1305 |
1305 |
1305 |
| the PR’s parser found no verdict |
0 |
0 |
0 |
0 |
0 |
0 |
1 |
7 |
507 |
Replies with no label line
| replies with no label |
1 |
7 |
503 |
| empty |
1 |
1 |
0 |
has extracted_final_answer |
0 |
6 |
488 |
has reasoning |
0 |
6 |
496 |
has confidence |
0 |
6 |
427 |
| verdict stated in words |
0 |
6 |
358 |
| median length (chars) |
0 |
340 |
300 |