Agent Evaluation
Human and LLM-judge evaluation of agent output: judge calibration, trace review, trajectory editing, programmatic evaluators, and CI evaluation.
This section treats a model's output as something to be reviewed rather than something to be labelled. The unit of work is usually a trace: one run of an agent, with its tool calls, intermediate steps and final answer.
LLM-as-Judge Calibration and Judge ↔ Human Alignment cover the case where a model does the grading and you need to know how far to trust it. Three-Pane Trace Evaluation covers the case where a person does. CI Evaluation covers running either on every commit.
Potato does not train models. Everything here produces labels, scores and agreement statistics; what you do with them afterwards happens outside the tool.