# Linguistic Fieldwork in Potato: Glossing with FLEx, Toolbox, Praat and ELAN Source: https://www.potatoannotator.com/docs/guides/linguistic-fieldwork **Potato 2.12.0 imports interlinear texts from [FieldWorks Language Explorer](https://software.sil.org/fieldworks/) (FLEx) and Toolbox, lets annotators review every sentence in an `interlinear_gloss` scheme that checks the Leipzig alignment rules, and exports the corrected glosses back to the same format. Praat TextGrid and ELAN files import the same way, as time-aligned tiers.** The analysis usually already lives in FLEx or Toolbox, and the work is checking and correcting it, often with several people reviewing different texts. Potato adds the review step: assignment of texts to reviewers, an alignment check on every word, and a record of who confirmed each sentence. ## Import FLEx or Toolbox texts ```bash # A FLEx interlinear export (FLExText) potato import -i stories.flextext -o glossing-project/ # A Toolbox or Shoebox text database potato import -i texts.txt -f toolbox -o glossing-project/ ``` `-i` can also be a directory of `.flextext` files, or of Toolbox files (`.txt`, `.sfm`, `.db`, `.tbx`, `.toolbox`). The format is detected from the content: a FLExText file has `` elements, and a Toolbox file has `\ref` records with a text line and a morpheme or gloss line. `-f flextext` or `-f toolbox` chooses explicitly. A FLExText file that cannot be read is left out with the reason: it is binary or not UTF-8 text, it is not well-formed XML, or it defines XML entities, which uploaded files may not. Each FLEx phrase or Toolbox record becomes one item. The existing analysis is written into the item's own fields (`words`, `morphemes`, `glosses`, `translation`, and `pos` when the source has parts of speech), and the generated scheme reads them as its starting point. Nothing counts as a person's answer until an annotator presses Next or edits the item, so an export before review contains only the sentences someone has checked. | FLExText | Potato | |---|---| | `interlinear-text` title | `text_title` field, used again on export | | `paragraph`, phrase `segnum` | `paragraph`, `phrase` fields | | word `txt` | A word. `punct` words stay in the sentence text but are not glossed | | morph `txt` (or `cf`) and `type` | The segmentation, with `-` or `=` taken from the form's own marks or from the type | | morph `gls` | The gloss of that morpheme, with spaces turned into `.` | | word `pos`, or the stem's `msa` | The `pos` tier | | phrase `gls` | The free translation | | vernacular and analysis `language` | The scheme's `language` and `metalanguage` | Toolbox records start at `\ref`, and the interlinear lines are read by column, as Toolbox writes them. Hand-edited files are often not aligned, so two readings by order are tried first: morphemes grouped into words by their own marks, and glosses paired with morphemes in order. The importer reads `\tx` or `\t` for the text, `\mb` or `\m` for morphemes, `\ge` or `\g` for the gloss, `\ps` or `\p` for part of speech, and `\ft` or `\f` for the free translation. Other markers in a record, such as `\nt`, are kept in the item's `sfm_extra` field and written back on export. ## Review the glosses ```bash python potato/flask_server.py start glossing-project/config.yaml -p 8000 ``` Annotators see each sentence with the imported analysis in place. Any word whose segmentation and gloss do not line up is marked when the page opens, so errors already in the source show on the first review. [Interlinear Glossed Text](/docs/annotation-types/interlinear-gloss) lists the checks and the keyboard controls. The checks follow the [Leipzig Glossing Rules](https://www.eva.mpg.de/lingua/resources/glossing-rules.php) by Bernard Comrie, Martin Haspelmath and Balthasar Bickel: - The example line and the gloss line are aligned word by word. - Within a word, `-` separates affixes and `=` separates clitics, and the gloss has the same separators in the same places, so morpheme and gloss can be paired by counting. - When one morpheme stands for several meanings, they are joined with a period: `NOM.SG`, `put.in.bag`. A period is not a boundary and is not counted. - Person and number may be written together without a period, as in `3SG`. - Grammatical categories are abbreviations in capitals, printed in small caps. - An infix is set off with angle brackets in both lines, and its gloss goes next to the stem's gloss. ## Export back to FLEx, Toolbox or LaTeX ```bash python -m potato.export --config glossing-project/config.yaml --format flextext --output out/ python -m potato.export --config glossing-project/config.yaml --format toolbox --output out/ python -m potato.export --config glossing-project/config.yaml --format igt --output out/ ``` - `flextext` writes `.flextext`, one per annotator, grouped into the texts and paragraphs the import recorded. Morph types are worked out from the boundaries: the first morpheme with a lowercase (lexical) gloss is the stem, morphemes before it are prefixes or proclitics, and those after it are suffixes or enclitics. - `toolbox` writes `.txt` with column-aligned lines and the original `\ref` values. - `igt` writes `igt.txt`, with three aligned lines and the translation per example, and `igt.tex` for the gb4e LaTeX package. Re-importing an exported file gives the same words, segmentations, glosses, translations and parts of speech. What does not survive the trip: - FLEx GUIDs and lexicon links, because the export has no lexicon. - FLEx punctuation words, which stay in the item's sentence text but are not written back as `punct` words. - The morph type of a morpheme that is neither the stem nor next to a marked boundary, such as a second root in a compound. - Tiers other than `pos` in FLExText. In Toolbox they are written as extra lines named after the tier. ## Praat TextGrids and ELAN files ```bash potato import -i recordings/ -o my-project ``` `recordings/` holds `.TextGrid` or `.eaf` files, each with an audio or video file of the same name beside it. Each file becomes an item with its media copied into `media/`, each tier becomes a tier of one `tiered_annotation` scheme, and every non-empty interval becomes a segment that annotators can correct. A tier with at most 30 distinct values, such as phones or speaker codes, gets them as its labels. The `textgrid` and `eaf` exporters write the corrected tiers back out. [Importing Projects](/docs/tools/importing-projects#praat-and-elan) has the details. ## Further Reading - [Interlinear Glossed Text](/docs/annotation-types/interlinear-gloss) - Scheme options, checks and the stored answer - [Transcript Formats](/docs/annotation-types/transcript-formats) - Time-aligned transcripts and tiers - [Perceptual Dialectology](/docs/guides/perceptual-dialectology) - Draw-a-map tasks for dialect perception - [Multilingual and Low-Resource Annotation](/docs/guides/multilingual-low-resource-annotation) - Annotating in many languages For implementation details, see the [source documentation](https://github.com/davidjurgens/potato/blob/master/docs/guides/linguistic-fieldwork.md).