Understanding Potato Data Formats

Potato reads input data and writes annotations in JSON and JSONL. There isn't much to it, but the details matter once your dataset grows or you start mixing text with images and audio. This guide walks through the formats with working examples for each data type.

Input data formats

JSON Lines (JSONL)

One JSON object per line. This is the format to reach for first:

json

{"id": "001", "text": "First document text here."}
{"id": "002", "text": "Second document text here."}
{"id": "003", "text": "Third document text here."}

Why JSONL: you can stream it line by line instead of loading the whole file into memory, appending a new record is just adding a line, and one malformed line won't take down the rest of the file.

JSON array

Standard JSON array:

json

[
  {"id": "001", "text": "First document."},
  {"id": "002", "text": "Second document."},
  {"id": "003", "text": "Third document."}
]

Configuration:

yaml

data_files:
  - data/items.json

Text Annotation Data

Basic Text

json

{"id": "doc_001", "text": "The product quality exceeded my expectations."}

With Metadata

json

{
  "id": "review_001",
  "text": "Great product, fast shipping!",
  "metadata": {
    "source": "amazon",
    "date": "2024-01-15",
    "author": "user123",
    "rating": 5
  }
}

With Pre-annotations

json

{
  "id": "ner_001",
  "text": "Apple announced new products in Cupertino.",
  "pre_annotations": {
    "entities": [
      {"start": 0, "end": 5, "label": "ORG", "text": "Apple"},
      {"start": 31, "end": 40, "label": "LOC", "text": "Cupertino"}
    ]
  }
}

Configuration:

yaml

data_files:
  - data/texts.json
 
item_properties:
  id_key: id
  text_key: text

Image Annotation Data

Local Images

json

{
  "id": "img_001",
  "image_path": "/data/images/photo_001.jpg",
  "caption": "Street scene in Paris"
}

Remote Images

json

{
  "id": "img_002",
  "image_url": "https://example.com/images/photo.jpg"
}

With Bounding Boxes

json

{
  "id": "detection_001",
  "image_path": "/images/street.jpg",
  "pre_annotations": {
    "objects": [
      {"bbox": [100, 150, 200, 300], "label": "person"},
      {"bbox": [350, 200, 450, 280], "label": "car"}
    ]
  }
}

Configuration:

yaml

data_files:
  - data/images.json
 
item_properties:
  id_key: id
  image_key: image_path  # or image_url

Audio Annotation Data

Local Audio

json

{
  "id": "audio_001",
  "audio_path": "/data/audio/recording.wav",
  "duration": 45.5,
  "transcript": "Hello, how are you today?"
}

With Segments

json

{
  "id": "audio_002",
  "audio_path": "/audio/meeting.mp3",
  "segments": [
    {"start": 0.0, "end": 5.5, "speaker": "Speaker1"},
    {"start": 5.5, "end": 12.0, "speaker": "Speaker2"}
  ]
}

Configuration:

yaml

data_files:
  - data/audio.json
 
item_properties:
  id_key: id
  text_key: transcript
 
instance_display:
  fields:
    - key: audio_path
      type: audio
      display_options:
        controls: true

The audio field is named by the display, not by item_properties. item_properties carries the id and text keys, and anything else an item needs travels in kwargs.

Multimodal Data

Text + Image

json

{
  "id": "mm_001",
  "text": "What is shown in this image?",
  "image_path": "/images/scene.jpg"
}

Text + Audio

json

{
  "id": "mm_002",
  "text": "Transcribe this audio:",
  "audio_path": "/audio/clip.wav",
  "reference_transcript": "Expected transcription here"
}

Output Annotation Format

Basic Output

json

{
  "instance_id": "doc_001",
  "user_id": "user123",
  "labels": {
    "sentiment": {"Positive": "true"},
    "confidence": {"5": "true"}
  },
  "spans": {},
  "links": {}
}

Span Annotations

Spans are grouped by scheme name. Each one stores its offsets and its label under name, and the jsonl exporter adds the text those offsets cover:

json

{
  "instance_id": "ner_001",
  "user_id": "user123",
  "labels": {},
  "spans": {
    "entities": [
      {"schema": "entities", "name": "ORG", "start": 0, "end": 5, "text": "Apple"},
      {"schema": "entities", "name": "PERSON", "start": 10, "end": 18, "text": "Tim Cook"},
      {"schema": "entities", "name": "LOC", "start": 27, "end": 32, "text": "Paris"}
    ]
  },
  "links": {}
}

Multiple Annotators

The export is one record per annotator per instance, so two annotators on the same item are two lines rather than one nested record. Potato writes no aggregated field; majority vote and agreement are things you compute from these rows:

json

{"instance_id": "item_001", "user_id": "ann1", "labels": {"sentiment": {"Positive": "true"}}, "spans": {}, "links": {}}
{"instance_id": "item_001", "user_id": "ann2", "labels": {"sentiment": {"Positive": "true"}}, "spans": {}, "links": {}}

Configuration Reference

yaml

data_files:
  - data/items.json
 
item_properties:
  id_key: id
  text_key: text
  image_key: image_path
 
instance_display:
  fields:
    - key: audio_path
      type: audio

A few habits that save headaches

Give every instance a unique ID; you will want it the moment you need to trace an annotation back to its source. Reach for JSONL once the file gets big. Validate the JSON before you load it, because a single stray comma fails late and confusingly. Keep metadata like source, date, and author around even if you don't use it now, since it makes debugging far easier later. And pick field names and stick to them, so downstream scripts don't need a special case for every file.

For the full list of supported keys, see the data format documentation.

Full data format documentation at Data Formats.