Medical Image Annotation with Potato

Potato's image annotation capabilities can be used for annotating medical images such as X-rays, photographs of skin lesions, or other clinical images in standard formats (PNG, JPG). This guide covers best practices for setting up medical image annotation projects.

Important Considerations

Before starting any medical annotation project:

De-identification: Ensure all PHI (Protected Health Information) is removed from images before annotation
Data handling: Use your institution's secure infrastructure for storing and processing medical data
Access control: Manage annotator access through your existing authentication systems
Compliance: Work with your IRB and compliance teams to meet institutional requirements

Note: Potato is an annotation tool and does not include built-in HIPAA compliance features, DICOM viewers, or specialized medical imaging capabilities. These requirements should be addressed at the infrastructure level.

Basic Medical Image Annotation Setup

Here's how to configure Potato for annotating standard medical images:

annotation_task_name: "Medical Image Annotation"
 
data_files:
  - path: data/medical_images.json
    type: json
 
annotation_schemes:
  - annotation_type: bounding_box
    name: lesions
    allow_multiple: true
    labels:
      - name: primary
        color: "#EF4444"
        description: "Primary finding"
 
      - name: secondary
        color: "#F97316"
        description: "Secondary finding"
 
      - name: artifact
        color: "#6B7280"
        description: "Artifact or noise"
 
  - annotation_type: radio
    name: image_quality
    question: "Overall image quality?"
    options:
      - name: diagnostic
        label: "Diagnostic Quality"
      - name: limited
        label: "Limited Quality"
      - name: non_diagnostic
        label: "Non-Diagnostic"

X-Ray and Clinical Photo Annotation

For annotating X-rays or clinical photographs that have been exported to standard image formats:

annotation_task_name: "X-Ray Findings Annotation"
 
data_files:
  - path: data/xray_images.json
    type: json
 
annotation_schemes:
  - annotation_type: polygon
    name: abnormalities
    allow_multiple: true
    labels:
      - name: opacity
        color: "#EF4444"
        description: "Pulmonary opacity"
 
      - name: consolidation
        color: "#F97316"
        description: "Consolidation"
 
      - name: nodule
        color: "#EAB308"
        description: "Pulmonary nodule"
 
      - name: effusion
        color: "#3B82F6"
        description: "Pleural effusion"
 
  - annotation_type: multiselect
    name: findings
    question: "Select all findings present"
    options:
      - name: normal
        label: "No acute findings"
      - name: pneumonia
        label: "Pneumonia"
      - name: atelectasis
        label: "Atelectasis"
      - name: pneumothorax
        label: "Pneumothorax"
      - name: fracture
        label: "Fracture"

Skin Lesion Annotation

For dermatology images:

annotation_task_name: "Dermoscopy Annotation"
 
data_files:
  - path: data/skin_lesions.json
    type: json
 
annotation_schemes:
  - annotation_type: polygon
    name: lesion_boundary
    labels:
      - name: lesion
        color: "#EF4444"
        description: "Lesion boundary"
 
  - annotation_type: multiselect
    name: dermoscopic_features
    question: "Select all features present"
    options:
      - name: pigment_network
        label: "Pigment Network"
      - name: dots_globules
        label: "Dots/Globules"
      - name: streaks
        label: "Streaks"
      - name: blue_white_veil
        label: "Blue-White Veil"
      - name: regression
        label: "Regression Structures"
      - name: vascular
        label: "Vascular Structures"
 
  - annotation_type: radio
    name: diagnosis
    question: "Most likely diagnosis?"
    options:
      - name: benign_nevus
        label: "Benign Nevus"
      - name: seborrheic_keratosis
        label: "Seborrheic Keratosis"
      - name: basal_cell
        label: "Basal Cell Carcinoma"
      - name: melanoma
        label: "Melanoma"
      - name: other
        label: "Other"
 
  - annotation_type: likert
    name: confidence
    question: "Diagnostic confidence"
    size: 5
    min_label: "Low"
    max_label: "High"

Retinal Image Annotation

For fundus photographs and other retinal images:

annotation_task_name: "Fundus Image Annotation"
 
data_files:
  - path: data/fundus_images.json
    type: json
 
annotation_schemes:
  - annotation_type: polygon
    name: anatomical_structures
    labels:
      - name: optic_disc
        color: "#FDE68A"
        description: "Optic disc boundary"
 
      - name: fovea
        color: "#A78BFA"
        description: "Fovea region"
 
      - name: macula
        color: "#93C5FD"
        description: "Macular region"
 
  - annotation_type: polygon
    name: pathology
    allow_multiple: true
    labels:
      - name: hemorrhage
        color: "#EF4444"
        description: "Retinal hemorrhage"
 
      - name: exudate
        color: "#FCD34D"
        description: "Hard/soft exudate"
 
      - name: microaneurysm
        color: "#F97316"
        description: "Microaneurysm"
 
  - annotation_type: radio
    name: dr_grade
    question: "Diabetic retinopathy grade"
    options:
      - name: none
        label: "No DR"
      - name: mild
        label: "Mild NPDR"
      - name: moderate
        label: "Moderate NPDR"
      - name: severe
        label: "Severe NPDR"
      - name: proliferative
        label: "PDR"

Data Format

Prepare your image data in JSON format:

[
  {
    "id": "case_001",
    "image": "images/case_001.png",
    "metadata": {
      "body_part": "chest",
      "modality": "xray"
    }
  },
  {
    "id": "case_002",
    "image": "images/case_002.png",
    "metadata": {
      "body_part": "chest",
      "modality": "xray"
    }
  }
]

Note: Images must be in standard web-compatible formats (PNG, JPG, etc.). DICOM files need to be converted to standard image formats before use with Potato.

Best Practices

Use qualified annotators: Medical annotation requires clinical expertise - ensure your annotators have appropriate training
Pre-process images: Convert specialized formats (DICOM, etc.) to standard image formats and ensure PHI is removed
Multiple annotators: Have multiple annotators label each image to measure inter-annotator agreement
Clear guidelines: Provide detailed annotation guidelines specific to your clinical domain
Quality checks: Include gold-standard cases to monitor annotator accuracy
Institutional compliance: Work with your compliance team to ensure data handling meets requirements

Limitations

Potato provides general-purpose image annotation and does not include:

Native DICOM file support or DICOM viewers
Whole slide image (WSI) viewers with multi-resolution zoom
Built-in HIPAA compliance or audit logging
Medical credential verification
Specialized radiology windowing tools

For these capabilities, consider preprocessing your data or integrating Potato with specialized medical imaging infrastructure.

For more on image annotation, see /docs/features/image-annotation.