Potato for healthcare and clinical annotation

Potato lets clinicians annotate entities and relations in clinical notes, segment CT and MR volumes from DICOM or NIfTI, and draw on whole-slide pathology images, with several readers per case and their agreement measured. Potato is free and open source and runs on your own servers, so clinical notes and images stay inside your institution.

Clinical annotation projects tend to share two requirements. The data cannot go to a cloud service, so the tool has to run on the institution's own infrastructure, and the annotators are scarce experts whose labels need checking against each other. Potato runs wherever you install the package, including on a network with no route out once setup is done. Potato assigns cases to annotators, keeps each annotator's labels, and reports agreement for every kind of label.

Clinical annotation tasks in Potato

Each row is an annotation type or display that ships with Potato 2.12, and the linked page has its configuration.

Patient information in DICOM headers

DICOM headers usually contain the patient's name, ID, birth date, and institution, so the volume viewer shows annotators only the header fields you list in display_tags. A field that the DICOM de-identification basic profile removes, such as PatientName or PatientID, is shown only if you also list the field in allow_identifying_tags. The server enforces the list, so a browser cannot request other fields. The raw DICOM and NIfTI files are not served for download unless you turn that on.

annotation_schemes:
  - annotation_type: volume_annotation
    name: ct_segmentation
    description: Outline the lesion and the bone ring
    source_field: volume
    labels:
      - {name: Lesion, color: "#e6553a", key_value: "1"}
      - {name: Bone, color: "#3a8ee6", key_value: "2"}
    default_window: Soft tissue
    display_tags: [Modality, BodyPartExamined, SliceThickness]

The filter cannot cover everything. File and directory names reach the browser, and PACS exports often name folders after the patient, so use pseudonymous names such as case_001/. A series flagged with burned-in annotation is refused, but pixels in other series are not inspected, and the files on disk stay as uploaded. De-identify the files before upload when the people running the server must not see identifying data either. Volume support needs pip install 'potato-annotation[volumes]', and whole-slide images need the wsi extra.

Agreement and export

Masks from volume_annotation are scored by 3D Dice per label between each pair of annotators, and a pair where neither painted a label is left out for that label rather than counted as agreeing. Masks export as NIfTI label volumes in the source geometry (volume_nifti) or as DICOM SEG objects that reference the source series (dicom_seg). Text annotations export as JSONL, CSV, CoNLL, or Parquet, and every annotator's labels are kept for adjudication.

Working alongside 3D Slicer, MedCAT, and REDCap

3D Slicer is a desktop platform for medical image analysis and detailed segmentation by one person, and MedCATtrainer is an interface for training a biomedical concept recognition and linking model. Potato fits when the same cases go to several readers through a browser and you need their agreement. Potato is not a clinical data capture system and has no audit trail of the kind REDCap provides for regulated studies, so keep identifiable health records in a system built for that purpose, and use Potato for the annotation.

Frequently asked questions

Can I annotate clinical notes containing PHI on our own servers?

Potato runs on servers you control and writes annotations to those servers' disks. Potato sends nothing to an outside service unless you configure one, such as a hosted AI endpoint. Whether a given deployment may store protected health information is a decision for your institution's privacy and security office, which will look at access controls, sign-in, encryption, and logging on the server as well as the application.

Does a self-hosted annotation tool make a project HIPAA compliant?

No tool makes a project compliant by itself. Compliance depends on how the system is deployed and operated, including access control, agreements, and safeguards on the server. Potato helps in two ways. First, Potato can run inside your institution's network with no outside connection once installed. Second, the volume viewer limits which DICOM header fields annotators see.

Can Potato open DICOM or NIfTI images, and what do the masks export to?

Yes, from version 2.12. A volume_annotation scheme opens a DICOM series or a .nii or .nii.gz file in three linked views, and an RTSTRUCT on the item can provide starting masks. Masks export as NIfTI label volumes or as DICOM SEG objects that reference the source series. 4D volumes, fused PET/CT, and oblique reformats are not supported.

How do I split imaging cases among readers and measure their agreement?

Set how many readers see each case with num_annotators_per_item, and Potato assigns the cases and keeps each reader's masks. The agreement report gives 3D Dice per label for each pair of readers on a case.

Further reading

• Medical image and clinical text annotation tools compared, for 3D Slicer, QuPath, MONAI Label, V7, and Encord beside Potato
• Air-gapped deployment, for what Potato serves locally
• SSO and OAuth, for signing annotators in through your institution's identity provider
• Life sciences and NLP and text, for neighboring fields on this site