# How to Annotate Threaded Conversations

Source: https://www.potatoannotator.com/docs/guides/annotating-threaded-conversations

**A forum thread flattened into a transcript loses the thing most conversational research is about: who was replying to whom.** Rendering reply structure is what makes a thread annotatable as a thread rather than as a list.

## Render the structure

```yaml
instance_display:
  fields:
    - key: conversation
      type: dialogue
      label: "Thread"
      display_options:
        indent_replies: true
```

Nesting derives from each turn's `reply_to`. Turn identity is read from `turn_id`, `step_id` or plain `id`, so forum exports, chat logs and mailing lists work without reshaping the data.

**Use stable ids.** Falling back to position means a re-fetched or re-ordered thread re-points every existing annotation, which is the kind of corruption that is invisible until someone checks.

## One field, several questions at once

The useful property is that a single conversation field can carry, simultaneously:

- **Whole-thread schemes** — did this thread go off the rails?
- **Per-comment radio, likert, select or text** — rate each comment
- **Spans inside comments** — mark the sentence that did it
- **`span_link` between spans in *different* comments** — link a claim to the reply that rebuts it

That last one is what most conversational annotation actually needs and what flat renderings cannot express. A rebuttal is a relation between two positions in two different turns.

For branching conversations, `conversation_tree` gains per-node widgets keyed by node id, so a branching tree and a flat thread can refer to the same messages.

## ConvoKit, both directions

[ConvoKit](https://convokit.cornell.edu/) (Cornell) is the standard corpus format for conversational research in CS and computational linguistics.

```bash
potato convokit conversations-gone-awry-corpus     # by name
potato convokit ./my-corpus/                       # by directory
potato convokit corpus.zip                         # by zip
```

Import at conversation or utterance granularity. Export back with `--format convokit`, either as `info.<field>.jsonl` overlays that drop into an existing corpus, or as a full dump.

Two properties worth knowing:

- **Each turn carries the real utterance id**, so per-comment annotations round-trip by direct lookup rather than by position.
- **No `convokit` dependency.** The format is read and written with the standard library, so importing a corpus does not pull in a research stack.

Every format variant in the wild is read, down to the pre-rename `user`/`root`/`users.json` layout.

## Measuring agreement

Thread-level labels use ordinary categorical agreement. Per-comment labels are the interesting case: agreement should be computed **per comment**, not pooled across the thread, or a long thread of easy comments will swamp the disagreement on the one comment that mattered.

Spans inside comments use [span agreement](/docs/guides/agreement-for-spans), with the caveat that offsets must be computed against the same normalisation the server applies. Potato had a bug here where the server collapsed whitespace and the client did not, drifting every offset in a dialogue.

## Further reading

- [Dialogue annotation](/docs/annotation-types/dialogue-annotation)
- [Conversation trees](/docs/annotation-types/conversation-trees)
- [Span linking](/docs/annotation-types/span-linking)
- [ConvoKit integration](https://github.com/davidjurgens/potato/blob/main/docs/integrations/convokit.md)