Gigapixel Image Annotation with Deep Zoom
Serve a tile pyramid as DZI and IIIF Image API 3.0, and paint brush masks at the source's full resolution with no texture-size ceiling.
Annotate images too large to send to a browser as one file — aerial surveys, stitched microscopy mosaics, scanned maps — with boxes, polygons and masks that work at any magnification. Potato serves a tile pyramid and layers the annotation canvas over it.
annotation_schemes:
- annotation_type: image_annotation
name: structures
description: "Mark every structure."
source_field: image
viewer: deepzoom # the default is `fabric`
tools: [bbox, polygon, brush, eraser]
labels:
- {name: building, color: "#c0392b"}media_directory is where the tile routes read from; paths in your data file are relative to it.
Runnable example: examples/image/deep-zoom/.
Gigapixel and Whole-Slide Image Annotation Tools Compared compares Potato with QuPath, ASAP, Cytomine, the Digital Slide Archive, napari, V7, and Recogito Studio on slide formats, licenses, and support for several annotators per slide. QuPath and ASAP suit one person outlining slides on a desktop, while Potato suits studies in which several annotators outline the same slides or mosaics.
Two protocols, one pyramid
The same tile pyramid is served as both DZI and IIIF Image API 3.0. IIIF matters for cultural-heritage and digital-humanities work, where an institution's existing viewers and citation practices already assume it. Rendering is by OpenSeadragon, vendored locally like every other asset.
Masks at full resolution
Brush masks work on tiled images at the source's full resolution, because the mask buffer indexes image pixels rather than a GPU texture. There is no texture-size ceiling to hit.
This is worth stating carefully, because an earlier version of this claim was wrong and was corrected. Other platforms do support masking on tiled images; V7 among them. What is specific here is the buffer design, not the capability.
The buffers are sparse and chunked, which is what makes this affordable: ten classes on a 12-megapixel image went from about 458 MB to about 46 MB, and mask data stopped scaling with image area at all, holding at 16 KB regardless of resolution.
Whole-slide images
From Potato 2.12, the deep-zoom viewer also opens pathology slides in the formats openslide reads (.svs, .ndpi, .mrxs, .scn, .vms, .vmu, .bif, and .svslide, plus any .tif that openslide recognizes as a slide). A slide already stores its own pyramid, so Potato cuts each tile from the nearest stored level the first time the viewer asks for the tile, and caches the result. Nothing is decoded whole, so a 100,000 × 80,000 pixel scan opens as quickly as a small image. Slide support needs an optional install, which bundles the openslide library:
pip install "potato-annotation[wsi]"The scheme is the same as for any deep-zoom image:
annotation_schemes:
- annotation_type: image_annotation
name: regions
description: "Outline each tissue region"
source_field: image
viewer: deepzoom
tools: [polygon, bbox, landmark]
labels: [tissue, "nuclei cluster", artifact]Coordinates are stored as fractions of the full-resolution slide, as for any image. The tile descriptor at /media/tiles/<slide>.dzi?format=json gives the slide's microns per pixel as mpp_x and mpp_y, so a box's width in microns is its width fraction times the slide width times mpp_x. The release includes a runnable example in examples/image/whole-slide/, with a script that generates a synthetic slide.
Dedicated pathology tools
For digital pathology, QuPath (desktop, open source, the field standard) and Cytomine (web, open source, multi-user with blind annotation) are built for pathology work. Potato fits when several annotators label the same slides and you need their agreement, or when the slides are one part of a larger study. CT and MR volumes in DICOM or NIfTI use a different scheme, volume_annotation, which the healthcare page describes.
Related
- Media ingest — 16-bit TIFF, HEIC, RAW
- Geometry primitives
- Gigapixel and Whole-Slide Image Annotation Tools Compared
- Train Detection and Segmentation Models on Image Annotations — detectors and segmenters trained on slides cut into tiles
- Source documentation
Example designs
Each design is a working Potato configuration you can download and run.
- BRACS Breast Lesion Regions on a Gigapixel Whole-Slide ImageOutline and label breast lesion regions with seven BRACS categories in the deep-zoom viewer, then train a tiled Mask R-CNN. The sample slide is illustrative.
- PANDA Gleason Pattern Regions and Grade Group on BiopsiesOutline benign tissue and Gleason patterns 3, 4 and 5 on a prostate biopsy in the deep-zoom viewer and choose the ISUP grade. The sample slide is illustrative.