Image Annotation

Annotate images in Potato with bounding boxes, polygons, freehand drawing, and landmark points. Includes zoom, pan, multi-label classification, and AI pre-labeling.

Image Annotation

Potato's image annotation system supports multiple annotation methods for computer vision tasks including object detection, segmentation, and keypoint annotation.

Supported Tools

Tool	Description	Use Case
`bbox`	Rectangular bounding boxes	Object detection
`polygon`	Multi-point polygons	Instance segmentation
`freeform`	Free-hand drawing	Irregular shapes
`landmark`	Point markers	Keypoint detection
`fill`	Pixel-level fill tool	Segmentation masks
`eraser`	Erase painted regions	Segmentation mask correction
`brush`	Paint brush tool	Fine-grained segmentation

Basic Configuration

yaml

annotation_schemes:
  - name: "objects"
    description: "Draw boxes around all objects"
    annotation_type: "image_annotation"
    tools:
      - bbox
    labels:
      - person
      - car
      - bicycle

Configuration Options

Field	Type	Default	Description
`name`	string	Required	Unique identifier for the annotation
`description`	string	Required	Instructions shown to annotators
`annotation_type`	string	Required	Must be `"image_annotation"`
`tools`	list	Required	Enabled annotation tools (`bbox`, `polygon`, `freeform`, `landmark`, `fill`, `eraser`, `brush`)
`labels`	list	Required	Category labels for annotations
`zoom_enabled`	boolean	true	Enable zoom controls
`pan_enabled`	boolean	true	Enable drag navigation
`min_annotations`	integer	0	Minimum required annotations
`max_annotations`	integer	null	Maximum allowed annotations (null = unlimited)
`freeform_brush_size`	integer	5	Brush size for freeform drawing
`freeform_simplify`	float	2.0	Path simplification factor

Label Configuration

Labels can be simple strings or detailed objects:

yaml

# Simple labels (auto-colored)
labels:
  - person
  - car
  - tree
 
# Detailed labels with custom colors and shortcuts
labels:
  - name: "person"
    color: "#FF6B6B"
    key_value: "1"
  - name: "car"
    color: "#4ECDC4"
    key_value: "2"
  - name: "bicycle"
    color: "#45B7D1"
    key_value: "3"

Examples

Object Detection (Bounding Boxes)

yaml

annotation_schemes:
  - name: "detection"
    description: "Draw bounding boxes around all vehicles"
    annotation_type: "image_annotation"
    tools:
      - bbox
    labels:
      - name: "car"
        color: "#3B82F6"
        key_value: "1"
      - name: "truck"
        color: "#10B981"
        key_value: "2"
      - name: "motorcycle"
        color: "#F59E0B"
        key_value: "3"
    min_annotations: 1
    zoom_enabled: true

Instance Segmentation (Polygons)

yaml

annotation_schemes:
  - name: "segmentation"
    description: "Draw polygons around each object"
    annotation_type: "image_annotation"
    tools:
      - polygon
    labels:
      - name: "building"
        color: "#8B5CF6"
      - name: "road"
        color: "#64748B"
      - name: "vegetation"
        color: "#22C55E"
    zoom_enabled: true
    pan_enabled: true

Keypoint Annotation (Landmarks)

yaml

annotation_schemes:
  - name: "facial_landmarks"
    description: "Mark facial keypoints"
    annotation_type: "image_annotation"
    tools:
      - landmark
    labels:
      - name: "left_eye"
        color: "#3B82F6"
      - name: "right_eye"
        color: "#3B82F6"
      - name: "nose"
        color: "#10B981"
      - name: "left_mouth"
        color: "#F59E0B"
      - name: "right_mouth"
        color: "#F59E0B"

Multiple Tools

yaml

annotation_schemes:
  - name: "mixed_annotation"
    description: "Use boxes for objects and polygons for regions"
    annotation_type: "image_annotation"
    tools:
      - bbox
      - polygon
    labels:
      - name: "object"
        color: "#3B82F6"
      - name: "region"
        color: "#10B981"

Keyboard Shortcuts

Key	Action
`b`	Activate bounding box tool
`p`	Activate polygon tool
`f`	Activate freeform tool
`l`	Activate landmark tool
`1-9`	Select label
`Delete`	Remove selected annotation
`Ctrl+Z`	Undo
`Ctrl+Y`	Redo
`+` / `-`	Zoom in/out

Data Format

Input Data

Your data file should include image paths or URLs:

json

[
  {
    "id": "img_001",
    "image_url": "https://example.com/images/photo1.jpg"
  },
  {
    "id": "img_002",
    "image_url": "/data/images/photo2.png"
  }
]

Configure the image field in your config:

yaml

item_properties:
  id_key: id
  text_key: image_url

Output Format

Annotations include geometry data for each annotation:

json

{
  "id": "img_001",
  "annotations": {
    "objects": [
      {
        "id": "ann_1",
        "type": "bbox",
        "label": "car",
        "geometry": {
          "x": 100,
          "y": 150,
          "width": 200,
          "height": 100
        }
      },
      {
        "id": "ann_2",
        "type": "polygon",
        "label": "building",
        "geometry": {
          "points": [
            [50, 50],
            [150, 50],
            [150, 200],
            [50, 200]
          ]
        }
      }
    ]
  }
}

Supported Image Formats

JPEG / JPG
PNG
GIF
WebP

Segmentation Masks

New in v2.2.0

The fill, eraser, and brush tools enable pixel-level segmentation mask annotation. These tools are ideal for semantic segmentation tasks where precise region boundaries are needed.

yaml

annotation_schemes:
  - name: "segmentation"
    description: "Paint segmentation masks"
    annotation_type: "image_annotation"
    tools:
      - fill
      - eraser
      - brush
    labels:
      - name: "foreground"
        color: "#3B82F6"
      - name: "background"
        color: "#6B7280"
    freeform_brush_size: 10

Key	Action
`i`	Activate fill tool
`e`	Activate eraser tool
`r`	Activate brush tool

PDF/Document Bounding Boxes

New in v2.2.0

Draw bounding boxes on PDF pages and document images. When combined with the pdf display type in instance_display, annotators can draw boxes on individual pages of PDF documents.

yaml

instance_display:
  fields:
    - key: pdf_url
      type: pdf
 
annotation_schemes:
  - name: "document_regions"
    description: "Draw boxes around figures and tables"
    annotation_type: "image_annotation"
    tools:
      - bbox
    labels:
      - name: "figure"
        color: "#3B82F6"
      - name: "table"
        color: "#10B981"
      - name: "equation"
        color: "#F59E0B"

Best Practices

Use appropriate tools - Bounding boxes for quick detection, polygons for precise boundaries, fill/brush for segmentation
Define clear labels - Use distinct, non-overlapping categories
Set annotation limits - Use min_annotations to ensure completeness
Enable zoom - Essential for detailed annotations on high-resolution images
Use keyboard shortcuts - Speeds up annotation significantly
Provide label colors - Makes it easier to distinguish annotations visually