# Agreement over Time

Source: https://www.potatoannotator.com/docs/measurement/temporal-agreement

**Two annotators marking when an event starts will not choose the same frame, so temporal agreement has to say how close counts as agreeing — and that choice changes the answer.** Potato reports the sweep rather than picking one threshold for you.

## What is measured

| Question | Measure |
|---|---|
| Do the segments overlap? | Temporal IoU |
| Do the boundaries land in the same place? | Boundary α |
| Did they mark the same *kind* of event? | Nominal α over labels |
| Did they agree there was an event at all? | Detection α |

## The tolerance sweep

For break-point and boundary annotations, agreement depends on a matching tolerance: how far apart can two marks be and still count as the same mark?

Reporting a single threshold invites the reader to assume whichever tolerance supports their conclusion. Agreement at 0.25 s and agreement at 2 s are different claims about the data:

- High at 2 s and low at 0.25 s means annotators agree an event happened but not precisely when. That is usually a guideline problem: the definition of "when it starts" is underspecified.
- Low at both means they disagree about whether it happened at all.

So the report sweeps the tolerance and shows the curve. See [world-model evaluation](/docs/vision-spatial/world-model-evaluation), where break-point agreement is additionally decomposed into detection, localization and category.

## Coverage is reported alongside

For dense curves — a progress-reward trace over a robot episode, for instance — agreement is reported as ICC plus Pearson **with coverage stated**.

A correlation computed over 5% of a timeline says nothing about the other 95%, and a coefficient that does not carry its coverage will be read as though it did.

## A known limitation

Temporal segments currently get the **uncorrected** measures only. σ's chance baseline is built from between-item distances, and "a segment from a different clip" is not a meaningful comparison when clips differ in length. Getting that right needs its own design, and claiming chance correction here before it exists would be worse than saying so.

## Related

- [Geometry agreement](/docs/measurement/geometry-agreement)
- [Robot episodes](/docs/vision-spatial/robot-episodes)
- [World-model evaluation](/docs/vision-spatial/world-model-evaluation)
- [Guide: agreement for spans](/docs/guides/agreement-for-spans)