Conversation Tree
Hierarchical conversation tree annotation with per-node quality ratings and path selection. Ideal for evaluating chatbot responses and dialogue systems.

Configuration Fileconfig.yaml
# Conversation Tree Annotation Configuration
# Demonstrates hierarchical tree annotation with per-node ratings
annotation_task_name: "Conversation Tree Evaluation"
task_dir: .
data_files:
- data.json
item_properties:
id_key: id
text_key: tree
user_config:
allow_all_users: true
instance_display:
fields:
- key: tree
type: conversation_tree
label: "Conversation"
collapsed_depth: 3
node_style: card
annotation_schemes:
- annotation_type: tree_annotation
name: response_quality
description: "Rate each response and select the best path through the conversation"
node_scheme:
annotation_type: likert
size: 5
min_label: Poor
max_label: Excellent
path_selection:
enabled: true
description: "Select the best response path by clicking nodes in order"
output_annotation_dir: annotation_output
output_annotation_format: json
Sample Datasample-data.json
[
{
"id": "conv_1",
"tree": {
"role": "user",
"content": "What is machine learning?",
"children": [
{
"role": "assistant",
"content": "Machine learning is a subset of artificial intelligence that enables systems to learn and improve from experience without being explicitly programmed.",
"children": [
{
"role": "user",
"content": "Can you give me an example?",
"children": [
{
"role": "assistant",
"content": "Sure! Email spam filters are a common example. The system learns to identify spam by analyzing thousands of emails marked as spam or not spam.",
"children": []
},
{
"role": "assistant",
"content": "A good example is recommendation systems like Netflix. They learn your preferences from your viewing history to suggest new shows.",
"children": []
}
]
}
]
},
{
"role": "assistant",
"content": "ML is when computers learn patterns from data. Think of it like teaching a computer by showing it examples instead of writing rules.",
"children": [
{
"role": "user",
"content": "How is it different from traditional programming?",
"children": [
{
"role": "assistant",
"content": "In traditional programming, you write explicit rules. In ML, the computer discovers the rules from data. For example, instead of writing rules to detect cats in images, you show the model thousands of cat images and it learns the patterns.",
"children": []
}
]
}
]
}
]
}
}
]Get This Design
This design is available in our showcase. Copy the configuration below to get started.
Quick start:
# Create your project folder mkdir conversation-tree cd conversation-tree # Copy config.yaml from above potato start config.yaml
Details
Annotation Types
Domain
Use Cases
Tags
Related Designs
PDTB 2.0 - Discourse Relations Tree Annotation
Discourse relation annotation with tree structure, based on the Penn Discourse TreeBank 2.0 (Prasad et al., LREC 2008). Annotators identify discourse connectives, mark argument spans, and build hierarchical discourse trees representing how text segments relate to each other.
RumourEval Stance and Verification
Annotate social media posts for rumour stance classification and thread structure analysis based on the RumourEval 2019 shared task. Annotators classify responses as supporting, denying, querying, or commenting on rumour claims.
Universal Dependencies - Dependency Parsing Annotation
Dependency parsing and POS tagging annotation based on Universal Dependencies v2 (Nivre et al., LREC 2020). Annotators build syntactic dependency trees and label parts of speech using the UD tagset.