Importing a Project from Label Studio, brat, doccano, Prodigy, CoNLL, Praat or a QDA Tool

Since Potato 2.9, one command turns a brat, doccano, Prodigy, CoNLL-2003, CoNLL-U or REFI-QDA .qdpx export into a runnable Potato project. The importer writes a data file, a config.yaml with the right annotation scheme already declared, and a README. The same command reads 16 computer-vision formats, covered in Computer Vision Formats. From Potato 2.12.0 it also reads TEI, 16 whole-project formats, Label Studio among them, and 7 survey formats (Survey Import), 45 in all.

potato import -i <your-export> -o my-project
potato start my-project/config.yaml -p 8000

The format is detected from the file. Pass --input-format only when detection cannot decide; it refuses to guess rather than produce a silently wrong import. potato import --list-formats lists everything supported.

Pre-annotations

Existing annotations arrive as a starting point. Every annotator sees them, and they are stored only once someone saves. An item nobody opens exports as empty, not as agreement nobody gave, which keeps machine output and old labels from being counted as new human work.

Supported text and qualitative formats

brat

Point at the directory, and the importer pairs each .ann with its .txt:

potato import -i ./my-corpus/ -o potato-project

brat offsets are Unicode code points with an exclusive end, which is Potato's convention too, so nothing is converted. Discontinuous mentions keep their extra fragments. Relations (R) and events (E) are listed in the warnings rather than imported, because they reference T ids that must be resolved to Potato span ids first, and resolving them wrongly would attach a relation to the wrong span without telling you.

doccano

potato import -i export.jsonl -o potato-project

Span entries are read in all three shapes doccano has emitted across versions ([start, end, label], start_offset/end_offset, and start/end), so an older export works. Document categories become their own multiselect scheme.

Prodigy

potato import -i db-out.jsonl -o potato-project

Tasks answered reject or ignore are not imported by default. A rejection is a person saying the annotation is wrong, and importing it as approved work turns that into positive signal. Pass --prodigy-keep-rejected to bring them in anyway. The answer field becomes a radio scheme, so the verdict survives either way.

CoNLL

potato import -i eng.train -o potato-project

CoNLL records tokens and their tags, never where they sat in a string, so the text is reconstructed before any character offset exists. How faithfully depends on what the file kept:

• CoNLL-U with # text = is exact, because the sentence string is in the file.
• CoNLL-U without it is rebuilt from tokens. SpaceAfter=No is honoured.
• CoNLL-2003 joins tokens with single spaces. The format records no spacing, so "Dr. Smith" and "Dr. Smith" produce the same file.

--conll-document-unit decides what counts as one item: auto (the default) follows -DOCSTART- and # newdoc markers and falls back to one item per sentence.

REFI-QDA (.qdpx)

.qdpx is the interchange format NVivo, ATLAS.ti, MAXQDA, Quirkos and QDA Miner all read and write.

potato import -i project.qdpx -o potato-project

The codebook comes across with its hierarchy, descriptions and colours, along with the codings and which annotator made each one. Potato also exports .qdpx, so a coded project can go back to the QDA tool it came from. See QDA Mode.

REFI-QDA 1.5 defines a selection by its first and last character, which makes the end position inclusive where Potato's is exclusive. Exporting tools have not all read the spec the same way, so the importer measures both readings against the source text, uses the one that fits, and warns when a file contradicts the spec. --qdpx-end-position inclusive|exclusive forces a reading.

ATLAS.ti keeps only one level of subcode. Export with --option flatten_subcodes=true to re-parent every code to the top level under a Parent > Child name, which loses the tree but keeps every code readable.

Whole projects

New in v2.12.0

Sixteen formats bring a whole project across: the questions, and where the source has them, the people who answered and the model suggestions beside each answer.

Label Studio

Export the project from Label Studio as Export → JSON (not JSON-MIN or CSV), and copy the labeling interface from Settings → Labeling Interface → Code into a file such as label_config.xml:

potato import -i project-export.json --ls-config label_config.xml -o my-project --keep-annotators
potato start my-project/config.yaml -p 8000

The XML is optional. Without it the questions are rebuilt from the answers, so every label someone used comes across but options nobody chose do not.

<Pairwise>, <Ranker>, <Taxonomy>, <BrushLabels>, relations, cancelled (skipped) annotations, predictions other than the highest-scoring one, and a second label on the same span are not imported. Each is reported in the output and the project README, with how many answers were dropped.

Keeping the source annotators

--keep-annotators works for Label Studio, Argilla, Zooniverse and W3C Web Annotation. It writes each source annotator's answers as that person's saved work under annotation_output/<name>/ and adds them to the roster, so /admin/iaa reports agreement as soon as the server starts. Unlike --seed-user it invents nobody, because the answers are real people's. It also sets max_annotations_per_item to the overlap the source study meant each item to get, so items the source study finished count as complete. Without the flag, only the model suggestions come across.

Praat and ELAN

potato import -i recordings/ -o my-project

recordings/ holds .TextGrid or .eaf files, each with an audio or video file of the same name beside it, such as interview01.TextGrid and interview01.wav. Each file becomes an item, and its media is copied into the project's media/. Each tier becomes a tier of one tiered_annotation scheme, and every non-empty interval becomes a segment on that tier. A tier with at most 30 distinct values, such as phones or speaker codes, gets them as its labels, and a tier of free text keeps each interval's text as the segment's value. TextGrids in UTF-16, as Praat often saves them, and in UTF-8 are both read, and the textgrid and eaf exporters write the corrected tiers back out. From 2.12.0, an EAF time slot with no time is interpolated from its neighbours instead of being read as 0.0.

Flags for research-tool formats

Without -f, potato import tries the survey importers first. A COCO file whose images carry seq_id imports as coco_camera_traps, an http(s) input goes to the IIIF importer, and tab-separated .txt files shaped like Raven or Audacity tables go to the bioacoustics importers. Pass -f to choose.

From 2.12.0, potato import does not copy or read local files from outside the input folder. An absolute path, a file:// URI, a ../ path or a symlink that leads out of the input is skipped with a warning, so an import cannot publish an unrelated file through media/. --image-dir and --texts folders are allowed.

Checking an import

Because imported annotations are pre-annotations, an import is hard to check end to end without opening every item. --seed-user writes them as one annotator's saved work, so you can export straight away and diff against the source:

potato import -i ./corpus/ -o p --seed-user check

--seed-user creates an annotator who never annotated anything. Use it only to verify the round trip, and keep that user out of agreement and adjudication.

Known gaps

Each of these produces a warning on import:

• brat relations and events are reported, not imported.
• Sub-token spans through CoNLL. A span covering Paris inside the token Paris. comes back as the whole token.
• Whitespace through CoNLL-2003, which the format does not record.
• Non-text QDA sources. Picture, PDF, audio and video sources in a .qdpx are not imported yet; text sources are.
• Uncoded selections in a .qdpx, which have no code to import. They are counted in the warnings.

Related

• Computer Vision Formats: the 16 CV importers
• Survey Import: Qualtrics, SurveyJS, LimeSurvey, XLSForm, REDCap, Beiwe and Network Canvas
• Export Formats: going the other way
• QDA Mode: what Potato does once a qualitative project is in
• Annotation Tools Compared

For implementation details, see the source documentation.