Docs/Annotation Types

Image Annotation

Annotate images with bounding boxes, polygons, freeform drawing, and landmark points.

Image Annotation

Potato's image annotation system supports multiple annotation methods for computer vision tasks including object detection, segmentation, and keypoint annotation.

Supported Tools

Tool	Description	Use Case
`bbox`	Rectangular bounding boxes	Object detection
`polygon`	Multi-point polygons	Instance segmentation
`freeform`	Free-hand drawing	Irregular shapes
`landmark`	Point markers	Keypoint detection
`fill`	Pixel-level fill tool	Segmentation masks
`eraser`	Erase painted regions	Segmentation mask correction
`brush`	Paint brush tool	Fine-grained segmentation

Basic Configuration

yaml

annotation_schemes:
  - name: "objects"
    description: "Draw boxes around all objects"
    annotation_type: "image_annotation"
    tools:
      - bbox
    labels:
      - person
      - car
      - bicycle

Configuration Options

Field	Type	Default	Description
`name`	string	Required	Unique identifier for the annotation
`description`	string	Required	Instructions shown to annotators
`annotation_type`	string	Required	Must be `"image_annotation"`
`tools`	list	Required	Enabled annotation tools (`bbox`, `polygon`, `freeform`, `landmark`, `fill`, `eraser`, `brush`)
`labels`	list	Required	Category labels for annotations
`zoom_enabled`	boolean	true	Enable zoom controls
`pan_enabled`	boolean	true	Enable drag navigation
`min_annotations`	integer	0	Minimum required annotations
`max_annotations`	integer	null	Maximum allowed annotations (null = unlimited)
`freeform_brush_size`	integer	5	Brush size for freeform drawing
`freeform_simplify`	float	2.0	Path simplification factor

Label Configuration

Labels can be simple strings or detailed objects:

yaml

# Simple labels (auto-colored)
labels:
  - person
  - car
  - tree
 
# Detailed labels with custom colors and shortcuts
labels:
  - name: "person"
    color: "#FF6B6B"
    key_value: "1"
  - name: "car"
    color: "#4ECDC4"
    key_value: "2"
  - name: "bicycle"
    color: "#45B7D1"
    key_value: "3"

Examples

Object Detection (Bounding Boxes)

yaml

annotation_schemes:
  - name: "detection"
    description: "Draw bounding boxes around all vehicles"
    annotation_type: "image_annotation"
    tools:
      - bbox
    labels:
      - name: "car"
        color: "#3B82F6"
        key_value: "1"
      - name: "truck"
        color: "#10B981"
        key_value: "2"
      - name: "motorcycle"
        color: "#F59E0B"
        key_value: "3"
    min_annotations: 1
    zoom_enabled: true

Instance Segmentation (Polygons)

yaml

annotation_schemes:
  - name: "segmentation"
    description: "Draw polygons around each object"
    annotation_type: "image_annotation"
    tools:
      - polygon
    labels:
      - name: "building"
        color: "#8B5CF6"
      - name: "road"
        color: "#64748B"
      - name: "vegetation"
        color: "#22C55E"
    zoom_enabled: true
    pan_enabled: true

Keypoint Annotation (Landmarks)

yaml

annotation_schemes:
  - name: "facial_landmarks"
    description: "Mark facial keypoints"
    annotation_type: "image_annotation"
    tools:
      - landmark
    labels:
      - name: "left_eye"
        color: "#3B82F6"
      - name: "right_eye"
        color: "#3B82F6"
      - name: "nose"
        color: "#10B981"
      - name: "left_mouth"
        color: "#F59E0B"
      - name: "right_mouth"
        color: "#F59E0B"

Multiple Tools

yaml

annotation_schemes:
  - name: "mixed_annotation"
    description: "Use boxes for objects and polygons for regions"
    annotation_type: "image_annotation"
    tools:
      - bbox
      - polygon
    labels:
      - name: "object"
        color: "#3B82F6"
      - name: "region"
        color: "#10B981"

Keyboard Shortcuts

Key	Action
`b`	Activate bounding box tool
`p`	Activate polygon tool
`f`	Activate freeform tool
`l`	Activate landmark tool
`1-9`	Select label
`Delete`	Remove selected annotation
`Ctrl+Z`	Undo
`Ctrl+Y`	Redo
`+` / `-`	Zoom in/out

Data Format

Input Data

Your data file should include image paths or URLs:

json

[
  {
    "id": "img_001",
    "image_url": "https://example.com/images/photo1.jpg"
  },
  {
    "id": "img_002",
    "image_url": "/data/images/photo2.png"
  }
]

Configure the image field in your config:

yaml

item_properties:
  id_key: id
  text_key: image_url

Output Format

Annotations include geometry data for each annotation:

json

{
  "id": "img_001",
  "annotations": {
    "objects": [
      {
        "id": "ann_1",
        "type": "bbox",
        "label": "car",
        "geometry": {
          "x": 100,
          "y": 150,
          "width": 200,
          "height": 100
        }
      },
      {
        "id": "ann_2",
        "type": "polygon",
        "label": "building",
        "geometry": {
          "points": [
            [50, 50],
            [150, 50],
            [150, 200],
            [50, 200]
          ]
        }
      }
    ]
  }
}

Supported Image Formats

JPEG / JPG
PNG
GIF
WebP

Segmentation Masks

New in v2.2.0

The fill, eraser, and brush tools enable pixel-level segmentation mask annotation. These tools are ideal for semantic segmentation tasks where precise region boundaries are needed.

yaml

annotation_schemes:
  - name: "segmentation"
    description: "Paint segmentation masks"
    annotation_type: "image_annotation"
    tools:
      - fill
      - eraser
      - brush
    labels:
      - name: "foreground"
        color: "#3B82F6"
      - name: "background"
        color: "#6B7280"
    freeform_brush_size: 10

Key	Action
`i`	Activate fill tool
`e`	Activate eraser tool
`r`	Activate brush tool

PDF/Document Bounding Boxes

New in v2.2.0

Draw bounding boxes on PDF pages and document images. When combined with the pdf display type in instance_display, annotators can draw boxes on individual pages of PDF documents.

yaml

instance_display:
  fields:
    - key: pdf_url
      type: pdf
 
annotation_schemes:
  - name: "document_regions"
    description: "Draw boxes around figures and tables"
    annotation_type: "image_annotation"
    tools:
      - bbox
    labels:
      - name: "figure"
        color: "#3B82F6"
      - name: "table"
        color: "#10B981"
      - name: "equation"
        color: "#F59E0B"

Best Practices

Use appropriate tools - Bounding boxes for quick detection, polygons for precise boundaries, fill/brush for segmentation
Define clear labels - Use distinct, non-overlapping categories
Set annotation limits - Use min_annotations to ensure completeness
Enable zoom - Essential for detailed annotations on high-resolution images
Use keyboard shortcuts - Speeds up annotation significantly
Provide label colors - Makes it easier to distinguish annotations visually

PreviousPure Display NextAudio Annotation