# Conversation Trees

Source: https://www.potatoannotator.com/docs/annotation-types/conversation-trees

The tree annotation schema enables annotation of hierarchical conversation structures such as chatbot response trees, dialogue systems, or branching narratives. Annotators can rate individual nodes, select preferred paths, and compare branches at decision points.

## Use Cases

- **Chatbot evaluation** - Rating quality of multiple response options
- **Dialogue systems** - Selecting preferred conversation paths
- **A/B testing** - Comparing different response strategies
- **Interactive fiction** - Evaluating branching story paths

## Quick Start

```yaml
annotation_schemes:
  - annotation_type: tree_annotation
    name: response_quality
    description: Evaluate the conversation tree
    node_scheme:
      annotation_type: likert
      min_label: "Poor"
      max_label: "Excellent"
      size: 5
    path_selection:
      enabled: true
      description: Select the best response path through the tree
```

## Configuration Options

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `annotation_type` | string | Required | Must be `"tree_annotation"` |
| `name` | string | Required | Unique identifier for this schema |
| `description` | string | Required | Instructions displayed to annotators |
| `node_scheme` | object | `{}` | Annotation scheme config for per-node annotation |
| `path_selection.enabled` | boolean | `false` | Enable path selection through the tree |
| `path_selection.description` | string | `"Select the best response path"` | Instructions for path selection |
| `branch_comparison.enabled` | boolean | `false` | Enable branch comparison mode |

## Examples

### Node Rating

Rate each response in the conversation tree:

```yaml
annotation_schemes:
  - annotation_type: tree_annotation
    name: response_rating
    description: Rate each response in the conversation
    node_scheme:
      annotation_type: likert
      min_label: "Very Bad"
      max_label: "Very Good"
      size: 5
```

### Path Selection

Select the best path through the conversation:

```yaml
annotation_schemes:
  - annotation_type: tree_annotation
    name: best_path
    description: Navigate the conversation tree
    path_selection:
      enabled: true
      description: Click on responses to build the best conversation path
```

### Multi-Criteria Node Rating

Rate nodes on multiple dimensions:

```yaml
annotation_schemes:
  - annotation_type: tree_annotation
    name: multi_criteria
    description: Evaluate each response on multiple criteria
    node_scheme:
      annotation_type: multirate
      options:
        - Relevance
        - Fluency
        - Helpfulness
      labels: ["1", "2", "3", "4", "5"]
```

### Branch Comparison Mode

Compare sibling branches at decision points:

```yaml
annotation_schemes:
  - annotation_type: tree_annotation
    name: branch_compare
    description: Compare response options at each decision point
    branch_comparison:
      enabled: true
    node_scheme:
      annotation_type: radio
      labels: ["Better", "Same", "Worse"]
```

## Data Format

### Input

Tree data should be provided in JSON format with a hierarchical structure:

```json
{
  "id": "conv_001",
  "tree": {
    "id": "root",
    "role": "user",
    "content": "Hello, I need help with my order",
    "children": [
      {
        "id": "resp_a",
        "role": "assistant",
        "content": "I'd be happy to help! Can you provide your order number?",
        "children": [
          {
            "id": "user_2",
            "role": "user",
            "content": "It's ORDER-12345",
            "children": []
          }
        ]
      },
      {
        "id": "resp_b",
        "role": "assistant",
        "content": "Sure, what seems to be the problem?",
        "children": []
      }
    ]
  }
}
```

Configure the tree data key:

```yaml
item_properties:
  id_key: id
  tree_key: tree
```

### Output

Tree annotations are saved with both node-level and path-level data:

```json
{
  "response_quality": {
    "node_annotations": {
      "resp_a": {
        "rating": 4
      },
      "resp_b": {
        "rating": 2
      }
    },
    "selected_path": ["root", "resp_a", "user_2"]
  }
}
```

## User Interface

### Tree Visualization

The conversation tree is displayed with nodes representing messages/responses, edges connecting parent-child relationships, and branching points where multiple responses exist.

### Node Selection

Click on any node to view the full message content, access the annotation panel, and add the node to the selected path (if path selection is enabled).

### Path Selection

When path selection is enabled, click nodes to add them to your path. The selected path is highlighted, and you can use "Clear Path" to start over.

## Best Practices

1. **Keep node schemes simple** - complex annotation interfaces on each node can slow annotation
2. **Provide context** - ensure annotators can see parent messages when rating a response
3. **Use path selection wisely** - works best for smaller trees
4. **Train annotators** - tree navigation requires practice; provide training examples

## Further Reading

- [Pairwise Comparison](/docs/annotation-types/pairwise-comparison) - For simpler A/B comparisons
- [AI Support](/docs/features/ai-support) - AI-assisted annotation

For implementation details, see the [source documentation](https://github.com/davidjurgens/potato/blob/main/docs/conversation_tree_annotation.md).