Skip to content
Showcase/Conversation Tree
intermediatepreference

Conversation Tree

Hierarchical conversation tree annotation with per-node quality ratings and path selection. Ideal for evaluating chatbot responses and dialogue systems.

Conversation Tree annotation interface preview

Configuration Fileconfig.yaml

yaml
# Conversation Tree Annotation Configuration
# Demonstrates hierarchical tree annotation with per-node ratings

annotation_task_name: "Conversation Tree Evaluation"
task_dir: .

data_files:
  - data.json

item_properties:
  id_key: id
  text_key: tree

user_config:
  allow_all_users: true

instance_display:
  fields:
    - key: tree
      type: conversation_tree
      label: "Conversation"
      collapsed_depth: 3
      node_style: card

annotation_schemes:
  - annotation_type: tree_annotation
    name: response_quality
    description: "Rate each response and select the best path through the conversation"
    node_scheme:
      annotation_type: likert
      size: 5
      min_label: Poor
      max_label: Excellent
    path_selection:
      enabled: true
      description: "Select the best response path by clicking nodes in order"

output_annotation_dir: annotation_output
output_annotation_format: json

Sample Datasample-data.json

json
[
  {
    "id": "conv_1",
    "tree": {
      "role": "user",
      "content": "What is machine learning?",
      "children": [
        {
          "role": "assistant",
          "content": "Machine learning is a subset of artificial intelligence that enables systems to learn and improve from experience without being explicitly programmed.",
          "children": [
            {
              "role": "user",
              "content": "Can you give me an example?",
              "children": [
                {
                  "role": "assistant",
                  "content": "Sure! Email spam filters are a common example. The system learns to identify spam by analyzing thousands of emails marked as spam or not spam.",
                  "children": []
                },
                {
                  "role": "assistant",
                  "content": "A good example is recommendation systems like Netflix. They learn your preferences from your viewing history to suggest new shows.",
                  "children": []
                }
              ]
            }
          ]
        },
        {
          "role": "assistant",
          "content": "ML is when computers learn patterns from data. Think of it like teaching a computer by showing it examples instead of writing rules.",
          "children": [
            {
              "role": "user",
              "content": "How is it different from traditional programming?",
              "children": [
                {
                  "role": "assistant",
                  "content": "In traditional programming, you write explicit rules. In ML, the computer discovers the rules from data. For example, instead of writing rules to detect cats in images, you show the model thousands of cat images and it learns the patterns.",
                  "children": []
                }
              ]
            }
          ]
        }
      ]
    }
  }
]

Get This Design

This design is available in our showcase. Copy the configuration below to get started.

Quick start:

# Create your project folder
mkdir conversation-tree
cd conversation-tree
# Copy config.yaml from above
potato start config.yaml

Details

Annotation Types

tree_annotation

Domain

NLPDialogue

Use Cases

Dialogue EvaluationChatbot AssessmentRLHF Data Collection

Tags

conversationtreedialoguechatbotevaluationRLHF