GoEmotions - Fine-Grained Emotion Classification
GoEmotions is a fine-grained emotion classification dataset built by Dorottya Demszky and colleagues at Google Research and published at ACL 2020.
Multi-label emotion classification with 27 emotion categories plus neutral, based on the Google Research GoEmotions dataset (Demszky et al., ACL 2020). Taxonomy covers 12 positive, 11 negative, and 4 ambiguous emotions designed for Reddit comment analysis.
Try it live — no install
Boot the real Potato server in your browser (WebAssembly) and annotate with this exact config. Nothing leaves your machine.
Run live in your browserGet This Design
Clone or download from the repository
Quick start:
git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/text/emotion-sentiment/goemotions
potato start config.yamlAbout this dataset
GoEmotions is a fine-grained emotion classification dataset built by Dorottya Demszky and colleagues at Google Research and published at ACL 2020. The authors wanted a large, manually annotated resource with a richer set of emotion labels than the handful used in most sentiment work, so that models could be trained and evaluated on subtle emotional distinctions.
The data is 58,009 English comments sampled from popular Reddit subreddits. Each comment was labeled for the emotions the author expresses, drawn from a taxonomy of 27 emotions plus a Neutral option. Annotators could assign more than one emotion to a comment, since a single message often carries several feelings at once.
The 27 emotions are grouped by sentiment into 12 positive, 11 negative, and 4 ambiguous categories. Each comment was rated by several annotators, and the authors report that 94% of examples have at least two raters agreeing on at least one emotion label; a filtered training split of 43,410 examples keeps only comments with rater agreement. A BERT baseline reaches an average F1 of .46 across the taxonomy.
The Potato config below reproduces this task with a multiselect scheme carrying all 27 emotion labels so annotators can mark every emotion present, plus a radio scheme that flags whether a comment is emotionally neutral. It suits anyone building an emotion or affect classifier who needs multi-label judgments on short, informal text.
- Comments
- 58,009 English Reddit comments
- Emotion categories
- 27 emotions + Neutral
- Sentiment grouping
- 12 positive, 11 negative, 4 ambiguous
- Inter-rater agreement
- 94% of examples have 2+ raters agreeing on a label
- Filtered training set
- 43,410 examples
- BERT baseline
- Average F1 = .46
Configuration File
This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.
# GoEmotions - Fine-Grained Emotion Classification
# Based on Demszky et al., ACL 2020
# Paper: https://aclanthology.org/2020.acl-main.372/
# Dataset: https://github.com/google-research/google-research/tree/master/goemotions
#
# Taxonomy: 27 emotions organized by sentiment:
# - Positive (12): admiration, amusement, approval, caring, desire, excitement,
# gratitude, joy, love, optimism, pride, relief
# - Negative (11): anger, annoyance, disappointment, disapproval, disgust,
# embarrassment, fear, grief, nervousness, remorse, sadness
# - Ambiguous (4): confusion, curiosity, realization, surprise
# - Plus: neutral
#
# Guidelines:
# - Select ALL emotions expressed in the text (multi-label)
# - Focus on emotion expressed by author, not evoked in reader
# - Related emotions differ by intensity (annoyance -> anger)
annotation_task_name: "GoEmotions: Fine-Grained Emotion Classification"
task_dir: "."
data_files:
- sample-data.json
item_properties:
id_key: "id"
text_key: "text"
output_annotation_dir: "annotation_output/"
export_annotation_format: "jsonl"
annotation_schemes:
- annotation_type: multiselect
name: emotions
description: "Select ALL emotions expressed in this text (multiple selections allowed)"
labels:
# Positive emotions (12)
- name: Admiration
tooltip: "Finding something impressive or worthy of respect"
key_value: "1"
- name: Amusement
tooltip: "Finding something funny or being entertained"
key_value: "2"
- name: Approval
tooltip: "Having or expressing a favorable opinion"
key_value: "3"
- name: Caring
tooltip: "Displaying kindness and concern for others"
key_value: "4"
- name: Desire
tooltip: "A strong feeling of wanting something"
- name: Excitement
tooltip: "Feeling very enthusiastic and eager"
- name: Gratitude
tooltip: "Being thankful, ready to show appreciation"
- name: Joy
tooltip: "A feeling of great pleasure and happiness"
key_value: "5"
- name: Love
tooltip: "A strong positive feeling of affection"
key_value: "6"
- name: Optimism
tooltip: "Hopefulness and confidence about the future"
- name: Pride
tooltip: "Pleasure or satisfaction from achievements"
- name: Relief
tooltip: "Reassurance and relaxation after anxiety"
# Negative emotions (11)
- name: Anger
tooltip: "A strong feeling of displeasure or hostility"
key_value: "7"
- name: Annoyance
tooltip: "Slight irritation or frustration"
- name: Disappointment
tooltip: "Sadness caused by unfulfilled expectations"
- name: Disapproval
tooltip: "Possession or expression of an unfavorable opinion"
- name: Disgust
tooltip: "A strong feeling of aversion or revulsion"
- name: Embarrassment
tooltip: "Self-conscious awkwardness or shame"
- name: Fear
tooltip: "Being afraid or worried about something"
key_value: "9"
- name: Grief
tooltip: "Deep sorrow, especially caused by loss"
- name: Nervousness
tooltip: "Easily agitated or worried"
- name: Remorse
tooltip: "Deep regret or guilt for wrongdoing"
- name: Sadness
tooltip: "Feeling or showing sorrow; unhappiness"
key_value: "8"
# Ambiguous emotions (4)
- name: Confusion
tooltip: "Lack of understanding; uncertainty"
- name: Curiosity
tooltip: "A strong desire to know or learn something"
- name: Realization
tooltip: "Becoming fully aware of something"
- name: Surprise
tooltip: "Feeling of astonishment or shock"
key_value: "0"
- annotation_type: radio
name: is_neutral
description: "Is this text emotionally neutral?"
labels:
- name: "Contains emotion(s)"
key_value: "e"
- name: "Neutral (no emotion)"
key_value: "n"
user_config:
allow_all_users: true
max_annotations_per_user: 200
num_annotators_per_item: 3
Sample Data
[
{
"id": "goemotions_001",
"text": "This is absolutely incredible! I can't believe how talented you are. Keep up the amazing work!"
},
{
"id": "goemotions_002",
"text": "Ugh, I've been waiting for 3 hours and still nothing. This is so frustrating."
}
]
// ... and 10 more itemsTry it live — no install
Boot the real Potato server in your browser (WebAssembly) and annotate with this exact config. Nothing leaves your machine.
Run live in your browserGet This Design
Clone or download from the repository
Quick start:
git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/text/emotion-sentiment/goemotions
potato start config.yamlDataset & paper
Demszky et al., ACL 2020
Citation (BibTeX)
@inproceedings{demszky-etal-2020-goemotions,
title = "{G}o{E}motions: A Dataset of Fine-Grained Emotions",
author = "Demszky, Dorottya and Movshovitz-Attias, Dana and Ko, Jeongwoo and Cowen, Alan and Nemade, Gaurav and Ravi, Sujith",
booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics",
month = jul,
year = "2020",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2020.acl-main.372",
pages = "4040--4054"
}Details
Annotation Types
Domain
Use Cases
Tags
Found an issue or want to improve this design?
Open an Issue(opens in a new tab)Related Designs
AfriHate - Hate and Abusive Language for African Languages
Multilingual content-moderation annotation following the AfriHate scheme (Muhammad et al., NAACL 2025), a collection of hate speech and abusive language datasets for 15 African languages including Amharic, Hausa, Igbo, Yoruba, Swahili, Somali and isiZulu. Native speakers familiar with the regional culture assign each tweet a three-way label - hate, abusive, or neutral - and, when the tweet is hateful, mark the targeted attribute (e.g. ethnicity, religion, politics, gender). Sample items are mild, constructed illustrations that avoid real slurs or targeting of real groups; an English gloss is provided for reference only.
iSarcasmEval: Intended Sarcasm Detection
SemEval-2022 Task 6 (iSarcasmEval): detecting intended sarcasm in English and Arabic. Unlike prior datasets labeled by third parties, iSarcasmEval texts are labeled by their own authors for intended sarcasm. Annotators decide whether a text is sarcastic and, if so, which category of ironic speech it expresses.
Moral Foundations in Tweets
Classification of moral foundations in social media discourse, based on Moral Foundations Theory (Johnson & Goldwasser, ACL 2018). Annotators identify which moral dimensions are expressed and whether the tweet conveys moral sentiment.