Gold-set labelling

One question per pair, answered by a person, from the records at their source

No file loaded.

The question is the same event — one thing that happened once, reported more than once. Not "similar". A batch of fifteen contaminated devices filed as fifteen reports is one event. Two similar problems on two occasions are two. Unclear is a real answer; use it rather than guessing. The method's own decision is in the file but is never shown here, on purpose.

1 · Load the pairs file

Pairs CSV from gold_sample.py

e.g. tools-data/maude/gold/LRC_pairs.csv. Read in this browser only.

Your initials (written into every row you label)

The second labeller uses a fresh copy of the file, without seeing the first's answers.

2 · The pair

Load a file to begin.

A draft is kept in this browser so a reload does not lose your answers; the downloaded file is the record.

3 · When you are done

Download the labelled file, commit it beside PROTOCOL.md as <CODE>_pairs.<initials>.csv, and run py -3 tools-data/maude/validate.py --code <CODE>. It measures agreement between the two labellers first, and prints precision and recall only when that clears the floor. The report template for what the numbers mean is docs/validation-report-template.md.