Transcript Formats

Potato reads 21 transcript and subtitle inputs and turns all of them into the same turn model before anything is displayed. Whatever your ASR pipeline or captioning tool produced, you can point Potato at it without writing a conversion script. This page is the reference for what works, how each format is recognized, and what each one carries. For the walkthrough, see Annotating Whisper Transcripts and Annotating YouTube Subtitles.

An SRT sidecar rendered as speaker bubbles, with a per-turn question attached to each

Potato does not transcribe

Speech recognition and speaker diarization happen before Potato sees anything. Run Whisper, WhisperX, pyannote, or a cloud API first; Potato ingests the result. Nothing on this page runs a model.

Note: Think-Aloud Mode does run a local Whisper model, but for an unrelated purpose. It records annotators talking while they work. That is a capture feature, not transcript ingestion.

Detection is by content, not file extension

Potato inspects what is inside the file rather than trusting its name. A WebVTT file called captions.txt still parses as WebVTT. The practical consequence is that the same transcript works either inlined into your data file or read from a sidecar on disk, with no configuration change.

Supported formats

ASR output

Subtitles and captions

SubRip and WebVTT cover most of what you will encounter. TTML shows up in broadcast archives and in files exported from professional captioning tools.

Alignment and linguistic annotation

Both the long and short TextGrid serializations parse. For ELAN files, Potato resolves ALIGNABLE_ANNOTATION and REF_ANNOTATION against the TIME_ORDER table and reads the media reference out of the header.

Each TextGrid tier becomes a speaker, and the empty intervals between them are skipped

Everything else

Three generic shapes round out the 21:

• A bare list of {speaker, start, end, text} dictionaries.
• An {"audio": ..., "turns": [...]} object.
• A plain paragraph with no timings at all, which renders as a single bubble.

Not supported

No parser exists for these. Convert them first.

• SAMI (.smi), MicroDVD (.sub), SubViewer (.sbv)
• Transcriber (.trs), EXMARaLDA, CHAT/CHILDES (.cha)
• Montreal Forced Aligner and Gentle native output. Export CTM or TextGrid from those instead.
• Azure Speech, Google Speech-to-Text, and Whisper API verbose_json inside unusual wrappers
• RTTM diarization files on their own

Word-level confidence is parsed and kept in the data model wherever the source provides it, but there is currently no interface for viewing it.

The normalized turn model

Every format above becomes this:

{
  "audio": "media/interview_01.mp3",
  "turns": [
    {
      "turn_id": "t0",
      "speaker": "host",
      "start": 0.0,
      "end": 6.5,
      "text": "Welcome back.",
      "words": [{"word": "Welcome", "start": 0.0, "end": 0.4, "confidence": 0.99}],
      "confidence": 0.97
    }
  ]
}

words and confidence appear only when the source carried them. speaker is null for undiarized turns, which render as Unassigned with a picker so annotators can fill them in.

YouTube auto-captions arrive with no speakers, so every turn is Unassigned until an annotator picks one

turn_id is the persistence key for per-turn annotations and speaker assignments. It comes from an explicit turn_id or step_id string in the source when there is one, otherwise t{index}. The same file always produces the same ids, so annotations survive a reload.

Warning: Time units differ between tools. Whisper and Deepgram use float seconds; AssemblyAI, whisper.cpp offsets, and Whisper's TSV output use integer milliseconds. Potato converts at the boundary so everything downstream is in seconds. If your own preprocessing mixes the two up, timings come out 1000× wrong.

Sidecar files

A field value that is a short single-line path ending in a known transcript extension gets read from disk instead of being treated as text. So the layout your ASR tool already produced, media files next to their .srt or .json, works without a preprocessing step:

{
  "id": "int_001",
  "conversation": {
    "audio": "media/int_001.mp3",
    "transcript": "media/int_001.srt"
  }
}

Paths resolve relative to task_dir and go through the same path-security validation as every other configured path, so a data file cannot read outside the project.

Recognized extensions: .srt .vtt .webvtt .json .json3 .srv1 .srv2 .srv3 .ttml .dfxp .xml .ass .ssa .tsv .ctm .TextGrid .eaf .txt

If your data genuinely holds one-line inline transcripts that happen to look like filenames, turn the heuristic off:

display_options:
  transcript_is_path: auto   # auto (default) | true | false

Configuration

Display: audio_dialogue

The audio dialogue display renders turns as speaker bubbles synced to the audio, with a play button on each turn.

instance_display:
  fields:
    - key: conversation
      type: audio_dialogue
      label: "Transcript"
      span_target: true
      display_options:
        audio_key: audio
        turns_key: turns
        speaker_key: speaker
        text_key: text
        transcript_is_path: auto
        show_timestamps: true
        allow_speaker_assignment: auto
        scroll_height: 460px

Schemes

As of 2.7.1, speech_transcript, voice_interaction, and tiered_annotation all read a transcript straight from the instance record and accept every format on this page. Before that, only the audio_dialogue display did.

annotation_schemes:
  - annotation_type: speech_transcript
    name: transcript_review
    description: "Mark transcription errors"
    segments_key: segments       # record field holding the transcript
 
  - annotation_type: voice_interaction
    name: barge_in
    description: "Mark overlaps and barge-in"
    turns_key: turns

Tiered annotation can pre-populate a tier from the transcript, so annotators correct an existing alignment rather than re-segmenting the speech by hand:

  - annotation_type: tiered_annotation
    name: tiers
    source_field: audio_url
    media_type: audio
    transcript_field: asr_output   # opt-in; omit to start from a blank timeline
    transcript_tier: utterance     # defaults to the first tier
    tiers:
      - name: utterance
        labels:
          - name: speech
            color: "#7c3aed"

Seeded annotations are not written until the annotator makes a real edit, so nothing gets attributed to someone who only opened the instance.

Building a data file with potato transcripts

Point the converter at a folder of ASR output and it writes an annotation-ready data file:

# Pair transcripts to media by basename
potato transcripts ./whisper_out --media-dir ./audio -o data/interviews.json
 
# Media served from elsewhere
potato transcripts './captions/*.vtt' \
  --media-url-prefix https://cdn.example.org/audio -o data/talks.json
 
# What did it detect? Writes nothing.
potato transcripts ./whisper_out --dry-run

--dry-run reports what it found without writing anything, which is the fastest way to catch a folder of the wrong Whisper output file:

Scanned 3 file(s):
  talk_01.srt          SRT                  2 turns      5.0s  2 speaker(s): Alice, Bob
  talk_02.vtt          WebVTT               1 turns      3.0s  undiarized
  talk_03.json         whisper.cpp JSON     1 turns      2.4s  undiarized

3 item(s), 4 turn(s).
1 item(s) have no media. Pass --media-dir or --media-url-prefix to enable playback.

Item ids come from the filename with a trailing media extension stripped, so Whisper's interview_01.mp3.json becomes interview_01 rather than interview_01.mp3.

Exporting back out

Tiered annotations export to ELAN EAF and Praat TextGrid, so a transcript can round-trip: ingest it, annotate in Potato, export, refine in ELAN or Praat, and read the result back in.

python -m potato.export --config config.yaml --format eaf --output ./out/
python -m potato.export --config config.yaml --format textgrid --output ./out/

Both exporters work from tiered_annotation output. Other schemes export through the standard formats.

Troubleshooting

Example project

examples/audio/transcript-formats/ in the Potato repository renders six formats side by side, each loaded from a sidecar file: SubRip, WebVTT, Whisper JSON, YouTube json3, Praat TextGrid, and Deepgram. All six produce the same speaker bubbles.

python potato/flask_server.py start examples/audio/transcript-formats/config.yaml -p 8000

The same task is available as a downloadable showcase design: Transcript Format Ingestion.

For implementation details, see the source documentation.

Further reading

• Annotating Whisper Transcripts, the end-to-end walkthrough
• Annotating YouTube Subtitles, for captions and caption-window caveats
• Dialogue Annotation, the speaker-bubble display
• Audio Annotation, waveform segmentation from scratch
• Designing Data Formats for Annotation