Image Annotation Tools Compared: CVAT, Label Studio, Roboflow, V7 and Potato

For high-volume computer-vision labelling, CVAT is the reference open-source tool, and Roboflow, V7 and Supervisely add model training or managed workflows as commercial products. Potato suits image labels that are research data: several annotators label the same images, and the study has to report how well they agree. Potato reports that agreement corrected for chance over boxes, polygons and masks, which we have not found in another annotation platform, and flags pre-labels accepted without review. It also fits when images share a study with text, audio or model output.

A bounding box is the rectangle around an object. Image segmentation assigns pixels to objects, as polygons or as masks. Keypoints mark landmarks such as joints, and a skeleton connects them. Intersection over union (IoU) measures how much two shapes overlap, and most tools use it to compare annotators.

This page covers images, gigapixel slides and 3D point clouds. Annotation Tools Compared covers every data type on one page.

Which image annotation tools are worth comparing?

Running a labelling study side by side

Every tool here draws the shapes. They differ on how they check the people drawing them:

A dash means we found nothing for it in the tool's documentation in September 2026. If a cell is wrong, tell us.

Where Potato is ahead for images

Agreement that corrects for chance. Two careless boxes on a large object still overlap heavily, so raw IoU flatters annotators. Potato compares annotators against an empirical chance baseline, and we have found no other annotation platform reporting chance-corrected agreement over spatial labels. The section on agreement below has the detail.

Pre-labels that were actually checked. Accepting a model's suggestion without looking raises agreement and lowers accuracy. Potato records drawing timing, which separates a reviewed suggestion from a rubber-stamped one.

The study around the images. Potato recruits annotators on Prolific or MTurk, trains them before the real task, and can add ratings, rankings or free-text questions about the same images to one study. CVAT's honeypots and Supervisely's exams check a labelling team. Potato's phases are built for study participants who arrive with no training.

Boxes, polygons and keypoints

Every tool here draws boxes and polygons. CVAT, Label Studio, Roboflow, V7, Supervisely, Segments.ai and Potato also do keypoints with skeletons. Potato adds polylines, ellipses, 2D cuboids and instance-keyed brush masks.

The mature computer-vision platforms attach attributes to each object, such as an occlusion level, a truncation flag or a sub-type. Potato does not yet. Geometry there carries a label, and extra attributes need a companion scheme.

Segmentation with a model's help

Interactive segmentation turns a click or a rough outline into a tight mask. Potato runs it in the browser, with no GPU to provision. CVAT reaches it through Nuclio functions or its AI-agent path, and Label Studio through an ML backend. Roboflow, V7, Supervisely and Segments.ai build it into the product.

Text-prompt segmentation returns a mask for an object you name. Potato runs it in the browser under an Apache-2.0 model. Roboflow and V7 use SAM 3 on their servers, Supervisely offers it through apps, and Label Studio through an ML backend. X-AnyLabeling bundles Grounding DINO, Grounded-SAM 2, SAM 2.1 and YOLO in a desktop app, and for one person labelling locally with many models it is hard to beat. Potato's advantage starts when there is more than one annotator.

See Image Segmentation Annotation and labelling objects by typing their name.

Gigapixel images and pathology slides

Potato builds a tile pyramid served as both DZI and IIIF Image API 3.0. Brush masks work at the source's full resolution, because the mask buffer indexes image pixels rather than a GPU texture, so there is no texture-size ceiling. 16-bit scientific TIFF gets a percentile window so faint structure stays visible. V7 and Supervisely also handle very large images, and CVAT partly does. See Gigapixel Image Annotation with Deep Zoom.

For digital pathology, QuPath (open-source desktop app, the field standard) and Cytomine (open-source web app with multi-user blind annotation) are purpose-built. Potato does not read SVS, DICOM or NIfTI.

3D point clouds

Potato annotates PCD, PLY, LAS, KITTI .bin and .xyz clouds with 3D cuboids, points, polylines and per-point segments. It stores cuboid rotation as a quaternion, so pitch and roll survive a round trip, and it compares cuboids with exact rotated 3D IoU. Annotation is per frame.

Segments.ai, Kognic, Deepen AI, Supervisely and Xtreme1/BasicAI are specialists, and for production autonomous-driving pipelines they are ahead: sequence workflows with track propagation, radar and multi-LiDAR support, and auto-fit cuboids. CVAT, Supervisely and Segments.ai handle point-cloud sequences, which Potato does not. See How to Annotate 3D Point Clouds.

Measuring agreement on image labels

Most computer-vision tools compare annotators by overlap. CVAT's consensus engine and V7's consensus stage compare annotators with raw IoU against a per-class threshold, and Label Studio Enterprise lists exact match, numeric difference, IoU and span overlap. None of these corrects for chance, and IoU between two boxes on a large object is high even when both are drawn carelessly.

Potato reports agreement over geometry with an empirical chance baseline. We have found no other annotation platform reporting chance-corrected agreement over spatial labels. How to Measure Inter-Annotator Agreement on Bounding Boxes explains the method.

Pre-labelling adds a second problem. Annotators who accept model suggestions without checking them raise every agreement number while lowering accuracy. Potato records drawing timing, which separates a reviewed suggestion from a rubber-stamped one. See Detecting Rubber-Stamped Pre-Labels.

Formats

Potato imports 15 computer-vision formats, 11 of which round-trip. It exports COCO, YOLO, Pascal VOC, CVAT, LabelMe, Cityscapes, KITTI, V7 Darwin, PNG masks, MOT and DAVIS. See Computer Vision Formats.

A bounding-box task with two annotators per image

agreement_metrics:
  enabled: true
 
annotation_schemes:
  - annotation_type: image_annotation
    name: objects
    description: "Draw a tight box around every vehicle."
    source_field: image
    tools: [bbox]
    labels:
      - {name: car, color: "#FF6B6B"}
      - {name: truck, color: "#4ECDC4"}
 
num_annotators_per_item:
  default: 2

Each image goes to two annotators, and the admin dashboard reports their agreement over the boxes.

What Potato does not do for images

• No model training. Potato pre-labels with existing models but does not train detectors. Roboflow, V7, Supervisely and Labelbox do.
• No per-object attributes yet.
• No point-cloud sequences. 3D annotation is per frame.
• No DICOM, NIfTI or whole-slide formats.
• No managed workforce. Bring your own annotators, or recruit them on Prolific.

Checked against each project's documentation and pricing page in August and September 2026.

Frequently asked questions

What is the best free image annotation tool?

For high-volume work that is only computer vision, CVAT is free, mature and self-hosted. For images in a project that also labels text or audio, Label Studio's Community Edition and Potato both cover the range. For studies in which several annotators label the same images and agreement has to be reported, Potato includes that at no cost. For a handful of images with nothing to install, VIA runs from a single HTML file.

Is there an open-source alternative to Labelbox?

CVAT, Label Studio's Community Edition and Potato are open source and self-hosted. Labelbox sells a managed platform and a labelling workforce. The open-source tools supply software, not annotators, so you bring your own team or recruit through a platform such as Prolific.

Can I run model-assisted segmentation without a GPU server?

Yes. Potato runs interactive and text-prompt segmentation in the browser through ONNX Runtime Web, and it works air-gapped once the model weights are on the machine. Its video mask propagation with SAM 2 runs on the server. X-AnyLabeling runs models locally on the desktop. CVAT and Label Studio call a model server.

How do I measure agreement between annotators on bounding boxes?

Assign each image to at least two annotators, match their boxes, and report agreement that accounts for chance, because raw IoU is high on large objects whatever the annotators do. The bounding-box agreement guide covers matching, the chance baseline and what to report.

Which annotation tool should I use for 3D point clouds?

For production driving data with sequences and track propagation, use a specialist such as Segments.ai, Kognic or Supervisely, or CVAT if it has to be open source. For per-frame cuboids where you need agreement statistics, or where 3D is one part of a multimodal study, Potato fits.

Is there an alternative to CVAT for annotation studies with several annotators?

For research in which each image goes to several annotators and the paper reports agreement, Potato is open source and free, and reports that agreement corrected for chance. CVAT's own quality tools, ground truth jobs, honeypots and consensus, score annotators by IoU against thresholds, with quality reports in CVAT Online and Enterprise. CVAT remains the stronger tool for high-volume labelling and point-cloud sequences.

Further reading

• Annotation Tools Compared, for every data type on one page
• Text Annotation Tools Compared
• Audio Annotation Tools Compared
• Video Annotation Tools Compared
• AI Agent Evaluation Tools Compared
• World Model and Robot Episode Evaluation Tools Compared
• Image Annotation