NuInsSeg Nucleus and Ambiguous Area Outlines in H&E Histology

# NuInsSeg (H&E-stained histology): outline nuclei and mark ambiguous areas
#
# Dataset: NuInsSeg, 665 H&E image patches from 31 human and mouse organs
# URL: https://zenodo.org/records/10518968
# Dataset DOI: 10.5281/zenodo.10518968
# Licence: CC BY 4.0 (dataset record and paper)
# Paper: Mahbod, A., Polak, C., Feldmann, K., Khan, R., Gelles, K., Dorffner, G.,
#   Woitek, R., Hatamikia, S., and Ellinger, I. (2024). NuInsSeg: A fully annotated
#   dataset for nuclei instance segmentation in H&E-stained histological images.
#   Scientific Data 11.
#   @article{Mahbod_2024, title={NuInsSeg: A fully annotated dataset for nuclei instance
#     segmentation in H\&E-stained histological images}, volume={11},
#     DOI={10.1038/s41597-024-03117-2}, number={1}, journal={Scientific Data},
#     author={Mahbod, Amirreza and Polak, Christine and Feldmann, Katharina and Khan, Rumsha
#     and Gelles, Katharina and Dorffner, Georg and Woitek, Ramona and Hatamikia, Sepideh
#     and Ellinger, Isabella}, year={2024}, month=Mar}
# DOI: 10.1038/s41597-024-03117-2
#
# Label glossary (Methods, "Generation of ground truth, auxiliary, and ambiguous
# area segmentation masks"):
#   nucleus: "We used the region of interest (ROI) manager tool ... and the freehand
#     option to delineate the nuclei borders. We manually draw the nuclei border for
#     each instance until all nuclei were segmented for a given image patch."
#   ambiguous area: "These vague areas consist of image parts with very complex
#     appearances where the accurate and reliable manual annotation is impossible."
#     Some images have none, for example the spleen image in the paper's Figure 1.
# Who annotated: "Manual segmentation of nuclei and ambiguous areas detection were
#   performed by three students with a background in cell biology. The annotations
#   were then controlled by a senior cell biologist and corrected when necessary."
#
# Sample items: ten real 512x512 tissue patches (human bladder, cerebellum, kidney,
# lung, melanoma, and tonsil; mouse femur, heart, liver, and spleen), copied
# unmodified from NuInsSeg.zip on Zenodo and shipped in media/ with attribution in
# media/LICENSE-media.txt. No masks are included.
#
# Adaptations:
#   - The original outlines were ImageJ freehand ROIs. Annotators here use the
#     polygon tool (or freeform) in the browser, so outlines are vertex polygons.
#   - Nuclei and ambiguous areas were two separate mask sets in the dataset. Here
#     they are two labels in one image_annotation scheme.
#   - The original review step by a senior cell biologist is not modeled. Potato
#     can compare two annotators' polygons at /admin/iaa, which gives a way to see
#     where outlines disagree.
#
# Model training (admin page /admin/training, admin key below):
#   cellpose fine-tunes Cellpose on the outlines annotators drew (each image's
#   outlines become an instance label image) and predicts polygons for images
#   nobody has outlined, written back as prelabels to correct. Cellpose has no
#   class, so it trains on every outline in the scheme, ambiguous areas included,
#   and gives every predicted instance the label set in trainer_params. Train it
#   once nuclei are the bulk of what was drawn, and check predictions near
#   ambiguous regions. It reads local image files, which is why the images are in
#   media/. Pretrained Cellpose weights come with the Cellpose authors' own
#   terms. Needs: pip install cellpose
#   sam-finetune is the other option for this scheme. It fine-tunes only SAM's
#   mask decoder, to turn boxes from an earlier detection run into outlines. It
#   does not touch SAM's image encoder and does not find nuclei by itself.
#   Needs: pip install 'potato-annotation[train-vision]'
#
# Run:
#   potato start config.yaml -p 8000

annotation_task_name: "NuInsSeg nuclei and ambiguous areas"

task_dir: "."
media_directory: media

data_files:
  - "sample-data.json"

item_properties:
  id_key: "id"
  text_key: "image_url"

output_annotation_dir: "annotation_output"
export_annotation_format: jsonl

user_config:
  allow_all_users: true
  users: []

require_password: false
admin_api_key: "demo-admin-key"

annotation_instructions: |
  Outline every nucleus in the H&E patch with one polygon each. Where nuclei are
  too crowded, blurred, or overlapped for a reliable outline, draw one polygon
  around the whole region and give it the ambiguous area label instead. Some
  patches have no ambiguous area. Zoom in on dense regions and pan by dragging.

annotation_schemes:
  - annotation_type: image_annotation
    name: nuclei
    description: "Outline each nucleus with a polygon. Outline regions where exact outlines are impossible with the ambiguous area label."
    source_field: image_url
    tools:
      - polygon
      - freeform
    labels:
      - name: nucleus
        color: "#E6194B"
        key_value: "1"
      - name: ambiguous area
        color: "#4363D8"
        key_value: "2"
    min_annotations: 0
    zoom_enabled: true
    pan_enabled: true

model_training:
  enabled: true
  writeback:
    schemas: ["nuclei"]
    include_unlabeled: true
    min_confidence: 0.5
  trainer_params:
    cellpose:
      base_model: nuclei
      epochs: 50
      label: nucleus