News Headline Emotion Roles (GoodNewsEveryone)
Annotate emotions in news headlines with semantic roles. Based on Bostan et al., LREC 2020. Identify emotion, experiencer, cause, target, and textual cue.
Configuration Fileconfig.yaml
This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.
# News Headline Emotion Roles (GoodNewsEveryone)
# Based on Bostan et al., LREC 2020
# Paper: https://aclanthology.org/2020.lrec-1.194/
#
# This task annotates emotions in news headlines with semantic roles:
# - Emotion: What emotion is conveyed?
# - Experiencer: Who feels the emotion?
# - Cause: What caused the emotion?
# - Target: What is the emotion directed at?
# - Cue: What words signal the emotion?
#
# Emotion Categories (Extended Plutchik):
# Joy, Sadness, Fear, Anger, Surprise (positive/negative),
# Disgust, Trust, Anticipation (positive/negative),
# Love, Pride, Guilt, Shame, Annoyance
#
# Annotation Guidelines:
# 1. First identify IF there's an emotion in the headline
# 2. Classify the emotion type
# 3. Identify semantic roles (may not all be present)
# 4. Consider both writer's emotion and reader's reaction
# 5. Headlines can evoke emotions even without explicit cues
annotation_task_name: "News Emotion Roles"
task_dir: "."
data_files:
- sample-data.json
item_properties:
id_key: "id"
text_key: "headline"
output_annotation_dir: "annotation_output/"
output_annotation_format: "json"
annotation_schemes:
# Step 1: Emotion classification
- annotation_type: radio
name: emotion
description: "What is the primary emotion in this headline?"
labels:
- "Joy"
- "Sadness"
- "Fear"
- "Anger"
- "Positive Surprise"
- "Negative Surprise"
- "Disgust"
- "Trust"
- "Positive Anticipation"
- "Negative Anticipation"
- "No emotion"
tooltips:
"Joy": "Happiness, delight, celebration"
"Sadness": "Grief, sorrow, disappointment"
"Fear": "Anxiety, worry, terror"
"Anger": "Frustration, outrage, annoyance"
"Positive Surprise": "Pleasant amazement, good news"
"Negative Surprise": "Shock, disbelief at bad news"
"Disgust": "Revulsion, disapproval"
"Trust": "Confidence, faith, security"
"Positive Anticipation": "Hope, excitement for future"
"Negative Anticipation": "Dread, pessimism"
"No emotion": "Neutral, factual headline"
# Step 2: Emotion intensity
- annotation_type: likert
name: intensity
description: "How intense is the emotion?"
min_value: 1
max_value: 5
labels:
1: "Very weak"
2: "Weak"
3: "Moderate"
4: "Strong"
5: "Very strong"
# Step 3: Mark emotion cue
- annotation_type: span
name: emotion_cue
description: "Highlight words that signal the emotion (if any)"
labels:
- "Emotion Cue"
label_colors:
"Emotion Cue": "#ef4444"
tooltips:
"Emotion Cue": "Words or phrases that indicate the emotion"
allow_overlapping: false
# Step 4: Mark cause
- annotation_type: span
name: cause
description: "Highlight what CAUSED the emotion (if present)"
labels:
- "Cause"
label_colors:
"Cause": "#3b82f6"
tooltips:
"Cause": "The event or entity that caused the emotion"
allow_overlapping: false
# Step 5: Reader vs writer emotion
- annotation_type: radio
name: perspective
description: "Whose emotion is this primarily?"
labels:
- "Writer/Subject's emotion"
- "Intended reader reaction"
- "Both"
- "Unclear"
tooltips:
"Writer/Subject's emotion": "The emotion of people in the story"
"Intended reader reaction": "The emotion the headline is meant to evoke in readers"
"Both": "Both writer and reader perspective"
"Unclear": "Cannot determine the perspective"
allow_all_users: true
instances_per_annotator: 50
annotation_per_instance: 3
allow_skip: true
skip_reason_required: false
Sample Datasample-data.json
[
{
"id": "gne_001",
"headline": "Local Hero Saves Family from House Fire"
},
{
"id": "gne_002",
"headline": "Unemployment Rate Hits Record Low"
}
]
// ... and 8 more itemsTry it live — no install
Boot the real Potato server in your browser (WebAssembly) and annotate with this exact config. Nothing leaves your machine.
▶ Run live in your browserGet This Design
Clone or download from the repository
Quick start:
git clone https://github.com/davidjurgens/potato-showcase.git cd potato-showcase/text/emotion-sentiment/news-emotion-roles potato start config.yaml
Dataset & paper
Bostan et al., LREC 2020
Citation (BibTeX)
@inproceedings{bostan-etal-2020-goodnewseveryone,
title = "{G}ood{N}ews{E}veryone: A Corpus of News Headlines Annotated with Emotions, Semantic Roles, and Reader Perception",
author = "Bostan, Laura Ana Maria and Kim, Evgeny and Klinger, Roman",
booktitle = "Proceedings of the 12th Language Resources and Evaluation Conference",
month = may,
year = "2020",
address = "Marseille, France",
publisher = "European Language Resources Association",
url = "https://aclanthology.org/2020.lrec-1.194",
pages = "1554--1566"
}Details
Annotation Types
Domain
Use Cases
Tags
Found an issue or want to improve this design?
Open an IssueRelated Designs
ESA: Error Span Annotation for Machine Translation
Error span annotation for machine translation output. Annotators identify error spans in translations, classify error types (accuracy, fluency, terminology, style), and rate severity.
LongEval: Faithfulness Evaluation for Long-Form Summarization
LongEval is the EACL 2023 protocol for human evaluation of faithfulness in long-form summaries (Krishna et al.). This Potato config reproduces its fine-grained, clause-level faithfulness judgments against source documents.
NLI with Explanations (e-SNLI)
Natural language inference with human explanations. Based on e-SNLI (Camburu et al., NeurIPS 2018). Classify entailment/contradiction/neutral and provide natural language justifications.