Text Annotation Tools Compared: Open-Source and Paid NLP Tools

A text annotation tool shows documents to annotators and records what they mark: a label for the whole text, highlighted spans such as named entities, and links between spans. Potato, INCEpTION, doccano and Label Studio's Community Edition are free to self-host, and all of them handle named-entity recognition and relations. They differ on coreference, knowledge-base linking, and whether agreement between annotators is measured without a paid plan. Prodigy is the paid option. brat and Argilla are no longer adding features.

Most text tasks take one of three forms. Text classification gives the whole document a label. Named-entity recognition and other span tasks mark stretches of text. Relation extraction and coreference resolution link spans to each other. Every tool on this page handles spans, and linking is where they differ most.

This page covers text only. Annotation Tools Compared covers every data type on one page.

Which text annotation tools are worth comparing?

Capabilities side by side

Agreement on text labels

Two annotators who mark the same entity rarely agree on its exact boundaries, and a metric that compares labels token by token counts every boundary difference as a disagreement about the label. How a tool scores span agreement matters as much as whether it reports agreement at all.

• Potato reports token-level κ over BIO tags, span F1 at exact and partial match, Krippendorff's α in its unitizing form, and γ from Mathet et al. (2015). The last two score where a span starts and stops as well as what it was called. See measuring agreement on spans.
• INCEpTION computes Cohen's κ, Fleiss' κ and Krippendorff's α, and has a curation interface for settling disagreements.
• Prodigy ships Krippendorff's α and Gwet's AC2, and a review recipe for adjudication.
• Label Studio keeps agreement for its paid plans. The metrics listed for its Enterprise edition are exact match, numeric difference, IoU and span overlap.
• doccano has no agreement metrics.

For the general case, see Inter-Annotator Agreement Explained.

The tools in more detail

INCEpTION

INCEpTION, from TU Darmstadt, succeeded WebAnno and is the closest comparison for linguistic work. It has relation layers, chain layers for coreference, document-level annotation, and concept features that link an annotation to an entry in a knowledge base such as Wikidata or DBpedia. On knowledge-base linking it is ahead of Potato. Its recommenders suggest annotations, with active learning built in. It is a Java application and takes more work to deploy than the other tools here. The system is described in Eckart de Castilho et al. (EMNLP 2024).

Prodigy

Prodigy, from Explosion, is built around annotation with a model in the loop and pairs with spaCy more closely than anything else here. It covers classification, spans, relations through its rel.manual recipe and coreference through coref.manual, and it reaches LLMs through spacy-llm. Tasks are defined in Python recipes. It is the one paid tool on this page.

Label Studio

Label Studio's text support is one part of a tool that also handles images, audio, video and time series. Its Relations tag links labelled regions. The Community Edition is free and open source. Agreement, ground-truth review and the quality dashboards are on the paid plans, starting with Starter at $99 a month.

doccano

doccano is the quickest of these to set up for text. Its project types are document classification, sequence labelling, sequence-to-sequence, intent detection and slot filling, and speech-to-text. Sequence-labelling projects can allow overlapping spans and relations. It has no coreference or event annotation and no agreement metrics.

brat

Many NLP corpora were built in brat, and its standoff format covers entities, n-ary events, binary relations, equivalence sets, attributes and normalizations. The last release is v1.3 from November 2012, and the last commit to master is from October 2021. Its standalone server imports Python's cgi module, which was removed in Python 3.13, so it will not start on a current Python without a shim. Reading a brat corpus is still worth doing. Starting a new project in brat is harder to justify.

Argilla

Argilla focuses on feedback and preference data for language models and integrates with Hugging Face Datasets. It supports text classification, token classification and evaluation of generated text. Its repository now says the original authors have moved on, and that the project will get bug fixes and patches but no new features.

Research systems

Several annotation systems have been published as ACL system demonstrations. Thresh configures fine-grained text-evaluation tasks in YAML, for summarization, simplification and machine translation. GATE Teamware 2 handles document classification with annotator training and quality screening.

Checked against each project's documentation, pricing page and repository in August and September 2026.

Moving a corpus between tools

Since version 2.9, Potato imports brat standoff, doccano JSONL, Prodigy db-out files and CoNLL, so a project started in any of them can continue in Potato. Existing labels arrive as pre-annotations. brat relations and events are listed in the import's warnings and are not imported. See Importing a Project.

Going the other way, Potato exports CoNLL-2003, CoNLL-U, Hugging Face datasets, JSONL, CSV and Parquet. See Exporting Annotations for ML.

An entity-and-relation task in Potato

annotation_schemes:
  - annotation_type: span
    name: entities
    description: "Highlight each person, organization and location."
    labels: [PERSON, ORGANIZATION, LOCATION]
  - annotation_type: span_link
    name: relations
    description: "Link each person to the organization they work for."
    span_schema: entities
    link_types:
      - name: WORKS_FOR
        directed: true
        allowed_source_labels: [PERSON]
        allowed_target_labels: [ORGANIZATION]

span_link draws typed links between spans from the entities scheme, and allowed_source_labels stops an annotator from linking the wrong kinds of entity. Relation and Event Extraction covers events and n-ary relations, and Coreference Resolution covers chains.

What Potato does not do for text

• Its knowledge-base linking is simpler than INCEpTION's.
• It does not update a spaCy model while annotators work, as Prodigy's recipes can.
• It has no spaCy exporter. Convert from CoNLL or JSONL.
• Importing from brat skips relations and events.

Frequently asked questions

What is the best free text annotation tool?

For a single NER or classification project, doccano is the least work to set up. For linguistic annotation that links entities to a knowledge base, INCEpTION is the most complete. For a study with several annotators per item, crowdworkers and an agreement figure to report, Potato includes all of that without a paid plan.

Is there an open-source alternative to Prodigy?

Potato, INCEpTION and doccano are free and open source. Potato is configured in YAML where Prodigy uses Python recipes, and it imports Prodigy's db-out files, so annotations already collected in Prodigy carry over.

Which text annotation tools report inter-annotator agreement?

Potato and INCEpTION do in their free versions, and Prodigy does in its paid product. Label Studio keeps agreement for its paid plans, and doccano has none. For span tasks, check whether the metric accounts for boundary disagreements, as Krippendorff's unitizing α and γ do.

Is brat still maintained?

No. Its last release was v1.3 in November 2012, and its standalone server does not start on Python 3.13 without a shim. From version 2.9, Potato imports brat corpora, with existing entities carried over as pre-annotations.

Can I annotate coreference in a free tool?

Yes. INCEpTION has chain layers for coreference, and Potato has a coreference scheme that groups mentions into chains. doccano does not support coreference.

Further reading

• Annotation Tools Compared, for every data type on one page
• Audio Annotation Tools Compared
• Image Annotation Tools Compared
• Video Annotation Tools Compared
• AI Agent Evaluation Tools Compared
• World Model and Robot Episode Evaluation Tools Compared
• Span Annotation
• Named Entity Recognition