यह पृष्ठ अभी आपकी भाषा में उपलब्ध नहीं है। अंग्रेज़ी संस्करण दिखाया जा रहा है।

How to Annotate Image Segmentation Masks

Polygons, brush masks and click-to-segment, when each is the right tool, how to write boundary guidelines annotators can follow, and how to measure agreement on a mask.

Segmentation labels which pixels belong to an object, and almost all of its difficulty is in the boundary rule rather than the drawing tool. Two annotators given the same image and no rule about shadows will disagree systematically, and no amount of tooling fixes that.

Polygon or mask

	Polygon	Brush mask
Holes and disconnected parts	Awkward	Natural
Thin structures (wires, hair, legs)	Poor	Good
Format support	Wide: COCO, YOLO, CVAT, LabelMe	Narrower: COCO RLE, DAVIS, mask PNG
Editability	Move a vertex	Repaint a region
Storage	Small	Large unless sparse

Polygons round-trip through more formats, which matters if the labels have to leave the tool. Masks are the honest answer when the object genuinely has holes or wisps.

If you are unsure, look at what the model consuming the labels needs. A semantic segmentation model consumes masks either way; the question is only what the annotators find least error-prone.

Write the boundary rule first

This is the part that determines your agreement number. Decide, and give examples for:

Shadow and reflection. In or out? Consistently.
Transparent and translucent parts. A glass, a windscreen, a plastic bag.
Thin structures. Where does a wire stop being worth tracing?
Occlusion. Mask only the visible part, or infer the whole object?
Contact. Two touching instances: where exactly is the split?
Motion blur. Where is the boundary of a blurred edge?

A single worked example per rule is worth more than a paragraph of prose. If annotators disagree after this, you will see it as low σ with high detection agreement — they are finding the same objects and drawing them differently, which is a guidelines problem rather than a carelessness problem.

Start from a model

Click-to-segment gives a first pass in roughly 130 ms, in the browser, with no GPU:

yaml

annotation_schemes:
  - annotation_type: image_annotation
    name: objects
    description: "Outline every object."
    source_field: image
    tools: [sam, polygon, brush, eraser]
    labels:
      - {name: object, color: "#6e56cf"}

bash

potato download-models onnxruntime
potato download-models mobile_sam

Positive clicks add to a mask, negative clicks cut away, a box constrains the search, and an existing mask can be refined.

The correction is the annotation value. If your annotators accept the first mask every time, you have collected a record of the model agreeing with itself — see telling review from rubber-stamping, because agreement statistics will not tell you.

Measuring agreement on masks

Do not use mean IoU between annotators. It answers "is this task easy", not "do these annotators agree" — see measuring agreement on bounding boxes for why.

Use STAPLE, which estimates a latent consensus boundary along with a sensitivity and specificity per annotator:

Low sensitivity means under-segmenting, drawing inside the true boundary.
Low specificity means over-segmenting, drawing generously around it.

They have opposite fixes, which is why a single accuracy number is not enough to give an annotator feedback.

STAPLE also weights by demonstrated reliability rather than counting votes. Two careful annotators outnumbered three-to-two by noisy ones score Dice 0.846 under majority vote and 1.000 under STAPLE.

Export

COCO carries polygons and RLE masks, including iscrowd crowd regions. DAVIS carries per-instance masks. YOLO carries polygons but not masks. Check the format matrix before committing to a mask workflow if the labels have to end up somewhere specific.