Video Annotation Tools Compared: CVAT, Label Studio, ELAN, BORIS and Potato

Video annotation is usually one of two jobs. Object tracking draws a shape around something and follows it across frames, and CVAT is the reference open-source tool for it. Temporal annotation marks when something happens, and ELAN and BORIS are the standards in linguistics and behavioural research. Potato does both in a browser for studies with several annotators. It recruits and trains those annotators, and reports how well they agree on events and on tracks.

Object tracking follows one object through a video, usually by drawing it on keyframes and interpolating the frames between them. A temporal segment is a start time, an end time and a label, such as an action or a speaker turn. Mask propagation uses a model to carry a segmentation mask from one frame to the next.

This page covers video. Annotation Tools Compared covers every data type on one page.

Which video annotation tools are worth comparing?

Running an annotation study side by side

A dash means we found nothing for it in the tool's documentation in September 2026. If a cell is wrong, tell us.

Where Potato is ahead for video

Agreement on both kinds of video annotation. ELAN and BORIS measure agreement on events, and CVAT on shapes. Potato does both: events are compared by overlap and detection at one matching threshold, and tracks and masks against a chance baseline, so a study that tracks objects and marks actions reports agreement on each.

Coders who were never in your lab. Potato recruits coders on Prolific or MTurk, trains them on practice clips with feedback, and checks them with gold clips as they work. Since version 2.9, each segment can carry its own rating or free-text question, so a coder can say why they marked an event where they did.

Tracking objects across frames

CVAT's track mode links shapes on different frames as one object. The annotator adjusts the shape on a few keyframes and CVAT interpolates the frames between them linearly, for rectangles, polygons and points. Label Studio's VideoRectangle tag adds object tracking to video, for boxes. V7, Supervisely and Segments.ai track with interpolation and propagate masks with a model.

Potato's video_annotation scheme has a tracking mode that interpolates boxes and polygons between keyframes. Mask propagation with SAM 2 runs on the server. The browser has a lighter carry-forward path. See How to Annotate Video Object Tracking.

Marking when things happen

Label Studio's TimelineLabels tag labels a single frame with a click or a range of frames with a drag. ELAN attaches annotations to the timeline on as many tiers as the scheme needs, with controlled vocabularies and up to four synchronized videos. BORIS logs and codes behavioural events from video, audio or live observation. Prodigy records regions against video timestamps, and VIA 3 annotates video as well as images and audio.

In Potato, temporal segments are labelled on a timeline. Since version 2.9, segment_schemes puts any annotation type inside a segment, so each segment can carry its own rating or free-text question. See Video Annotation.

Agreement on video annotations

Two annotators marking when an event starts will not pick the same frame, so temporal agreement depends on how far apart two marks can be and still count as the same one.

• ELAN has a built-in inter-annotator reliability calculation: a modified Cohen's κ that links annotations overlapping by a set minimum, and the Staccato algorithm, which accounts for chance by comparing a segmentation with randomly generated ones.
• BORIS computes Cohen's κ between two observations, matching events within a time window that defaults to ten seconds.
• Potato compares every pair of annotators on temporal IoU, detection F1 and an overall agreement score that gives half credit to matched segments with different labels, all at one matching threshold. None of these has a chance baseline yet, and its documentation says so. See Agreement over Time.
• CVAT's consensus engine compares annotators with raw IoU against a per-class threshold.

For tracks and masks, Potato compares masks frame by frame with mask consensus and STAPLE, and boxes with the geometry agreement described in How to Measure Inter-Annotator Agreement on Bounding Boxes.

Formats

Potato exports MOT for tracks, DAVIS for video object segmentation masks, and EAF for time-aligned annotations that ELAN opens. See Computer Vision Formats.

A tracking task in Potato

annotation_schemes:
  - annotation_type: video_annotation
    name: tracks
    description: "Draw the object once, then press Track forward."
    source_field: video_url
    mode: tracking
    labels:
      - {name: subject, color: "#d1495b"}
    tracking_options:
      interpolation: linear
      auto_advance_frames: 5
    video_fps: 12
    frame_stepping: true

frame_stepping lets annotators move one frame at a time, and auto_advance_frames sets how far Track forward jumps.

What Potato does not do for video

• No per-object attributes yet. A track carries a label. Occlusion and truncation flags need a companion scheme.
• No in-browser mask propagation. SAM 2 propagation runs on the server.
• No managed workforce. Bring your own annotators, or recruit them on Prolific.

Checked against each project's documentation in August and September 2026.

Frequently asked questions

What is the best free video annotation tool?

For tracking objects at volume, CVAT is free, mature and self-hosted. For coding behaviour or speech over time, ELAN and BORIS are free desktop tools. For a study in which several annotators label the same videos in a browser and agreement has to be reported, Potato includes that at no cost.

Which video annotation tools interpolate between keyframes?

CVAT's track mode interpolates rectangles, polygons and points linearly between keyframes. Potato's tracking mode interpolates boxes and polygons. V7 and Supervisely interpolate and also propagate masks with a model. Label Studio's VideoRectangle tracks boxes.

What should I use for behavioural coding of video?

BORIS and ELAN are the established free tools, and both compute inter-rater reliability: ELAN between pairs of tiers, BORIS as Cohen's κ between two observations. For a coding scheme applied by several coders in a browser, with coders recruited and trained online, Potato's temporal segments work. Its agreement report on segments gives overlap and detection at one matching threshold, without chance correction.

Which video annotation tools calculate inter-rater reliability?

ELAN offers a modified Cohen's κ and the Staccato algorithm between pairs of tiers. BORIS computes Cohen's κ between two observations within a time window. Potato reports temporal IoU and detection F1 on segments, without chance correction, and agreement on tracks and masks against a chance baseline. CVAT compares annotators' shapes by IoU against a threshold, and Label Studio keeps agreement for its paid plans.

How is agreement measured on video annotations?

For events, match each annotator's segments within a tolerance and report overlap, boundary agreement and label agreement, preferably across several tolerances, because agreement at a quarter of a second and at two seconds are different claims. For tracks, compare shapes frame by frame, with IoU for boxes and masks.

Where does video mask propagation run in Potato?

On the server, using SAM 2. The browser has a lighter carry-forward path that copies a mask to the next frame without the model.

Further reading

• Annotation Tools Compared, for every data type on one page
• Text Annotation Tools Compared
• Audio Annotation Tools Compared
• Image Annotation Tools Compared
• AI Agent Evaluation Tools Compared
• World Model and Robot Episode Evaluation Tools Compared
• How to Annotate Video Object Tracking