Video Annotation Tools Compared: CVAT, Label Studio, ELAN, BORIS and Potato

Video annotation is usually one of two jobs. Object tracking draws a shape around something and follows it across frames, and CVAT is the reference open-source tool for it. Temporal annotation marks when something happens, and ELAN and BORIS are the standards in linguistics and behavioural research. Potato does both in a browser for studies with several annotators, and reports how well they agree on when events start and end.

Object tracking follows one object through a video, usually by drawing it on keyframes and interpolating the frames between them. A temporal segment is a start time, an end time and a label, such as an action or a speaker turn. Mask propagation uses a model to carry a segmentation mask from one frame to the next.

This page covers video. Annotation Tools Compared covers every data type on one page.

Which video annotation tools are worth comparing?

Tracking objects across frames

CVAT's track mode links shapes on different frames as one object. The annotator adjusts the shape on a few keyframes and CVAT interpolates the frames between them linearly, for rectangles, polygons and points. Label Studio's VideoRectangle tag adds object tracking to video, for boxes. V7, Supervisely and Segments.ai track with interpolation and propagate masks with a model.

Potato's video_annotation scheme has a tracking mode that interpolates boxes and polygons between keyframes. Mask propagation with SAM 2 runs on the server. The browser has a lighter carry-forward path. See How to Annotate Video Object Tracking.

Marking when things happen

Label Studio's TimelineLabels tag labels a single frame with a click or a range of frames with a drag. ELAN attaches annotations to the timeline on as many tiers as the scheme needs, with controlled vocabularies and up to four synchronized videos. BORIS logs and codes behavioural events from video, audio or live observation. Prodigy records regions against video timestamps, and VIA 3 annotates video as well as images and audio.

In Potato, temporal segments are labelled on a timeline. Since version 2.9, segment_schemes puts any annotation type inside a segment, so each segment can carry its own rating or free-text question. See Video Annotation.

Agreement on video annotations

Two annotators marking when an event starts will not pick the same frame, so temporal agreement depends on how far apart two marks can be and still count as the same one.

• ELAN has a built-in inter-annotator reliability calculation: a modified Cohen's κ that links annotations overlapping by a set minimum, and the Staccato algorithm, which accounts for chance by comparing a segmentation with randomly generated ones.
• Potato reports temporal IoU for overlap, α for where boundaries fall, α over labels, and α for whether an event was marked at all, with the matching tolerance shown as a sweep across values. Its documentation lists one limitation: segment overlap does not yet have a chance baseline. See Agreement over Time.
• CVAT's consensus engine compares annotators with raw IoU against a per-class threshold.

For tracks and masks, Potato compares masks frame by frame with mask consensus and STAPLE, and boxes with the geometry agreement described in How to Measure Inter-Annotator Agreement on Bounding Boxes.

Formats

Potato exports MOT for tracks, DAVIS for video object segmentation masks, and EAF for time-aligned annotations that ELAN opens. See Computer Vision Formats.

A tracking task in Potato

annotation_schemes:
  - annotation_type: video_annotation
    name: tracks
    description: "Draw the object once, then press Track forward."
    source_field: video_url
    mode: tracking
    labels:
      - {name: subject, color: "#d1495b"}
    tracking_options:
      interpolation: linear
      auto_advance_frames: 5
    video_fps: 12
    frame_stepping: true

frame_stepping lets annotators move one frame at a time, and auto_advance_frames sets how far Track forward jumps.

What Potato does not do for video

• No per-object attributes yet. A track carries a label. Occlusion and truncation flags need a companion scheme.
• No in-browser mask propagation. SAM 2 propagation runs on the server.
• No managed workforce. Bring your own annotators, or recruit them on Prolific.

Checked against each project's documentation in August and September 2026.

Frequently asked questions

What is the best free video annotation tool?

For tracking objects at volume, CVAT is free, mature and self-hosted. For coding behaviour or speech over time, ELAN and BORIS are free desktop tools. For a study in which several annotators label the same videos in a browser and agreement has to be reported, Potato includes that at no cost.

Which video annotation tools interpolate between keyframes?

CVAT's track mode interpolates rectangles, polygons and points linearly between keyframes. Potato's tracking mode interpolates boxes and polygons. V7 and Supervisely interpolate and also propagate masks with a model. Label Studio's VideoRectangle tracks boxes.

What should I use for behavioural coding of video?

BORIS and ELAN are the established free tools, and ELAN can compute inter-annotator reliability between pairs of tiers. For a coding scheme applied by several coders in a browser, with agreement reported across them, Potato's temporal segments work.

How is agreement measured on video annotations?

For events, match each annotator's segments within a tolerance and report overlap, boundary agreement and label agreement, preferably across several tolerances, because agreement at a quarter of a second and at two seconds are different claims. For tracks, compare shapes frame by frame, with IoU for boxes and masks.

Where does video mask propagation run in Potato?

On the server, using SAM 2. The browser has a lighter carry-forward path that copies a mask to the next frame without the model.

Further reading

• Annotation Tools Compared, for every data type on one page
• Text Annotation Tools Compared
• Audio Annotation Tools Compared
• Image Annotation Tools Compared
• AI Agent Evaluation Tools Compared
• World Model and Robot Episode Evaluation Tools Compared
• How to Annotate Video Object Tracking