Guides
Tool-agnostic explainers on annotation practice: scheme design, inter-annotator agreement, crowdsourcing, and agent evaluation, each with a worked example.
These are explainers rather than reference pages. Each one answers a question about annotation practice that is not specific to any tool, then uses Potato as the worked example.
They fall into a few groups. The first cover decisions you make before any data is labelled: what to annotate, how to write guidelines, what shape the data should be in. A larger group covers particular kinds of annotation, from named entities and coreference through audio, video, image segmentation and 3D point clouds. A third covers measurement: agreement, adjudication, sample size, and what to do when annotators disagree. The rest cover evaluating LLM and agent output, and running a study with paid annotators.
If you are new to annotation, start with What Is Data Annotation?. If you have annotations already and need to report on their quality, start with Inter-Annotator Agreement Explained.