Skip to content
Showcase/GoEmotions - Fine-Grained Emotion Classification
intermediatetext

GoEmotions - Fine-Grained Emotion Classification

Multi-label emotion classification with 27 emotion categories plus neutral, based on the Google Research GoEmotions dataset (Demszky et al., ACL 2020). Taxonomy covers 12 positive, 11 negative, and 4 ambiguous emotions designed for Reddit comment analysis.

About this dataset

GoEmotions is a fine-grained emotion classification dataset built by Dorottya Demszky and colleagues at Google Research and published at ACL 2020. The authors wanted a large, manually annotated resource with a richer set of emotion labels than the handful used in most sentiment work, so that models could be trained and evaluated on subtle emotional distinctions.

The data is 58,009 English comments sampled from popular Reddit subreddits. Each comment was labeled for the emotions the author expresses, drawn from a taxonomy of 27 emotions plus a Neutral option. Annotators could assign more than one emotion to a comment, since a single message often carries several feelings at once.

The 27 emotions are grouped by sentiment into 12 positive, 11 negative, and 4 ambiguous categories. Each comment was rated by several annotators, and the authors report that 94% of examples have at least two raters agreeing on at least one emotion label; a filtered training split of 43,410 examples keeps only comments with rater agreement. A BERT baseline reaches an average F1 of .46 across the taxonomy.

The Potato config below reproduces this task with a multiselect scheme carrying all 27 emotion labels so annotators can mark every emotion present, plus a radio scheme that flags whether a comment is emotionally neutral. It suits anyone building an emotion or affect classifier who needs multi-label judgments on short, informal text.

Comments
58,009 English Reddit comments
Emotion categories
27 emotions + Neutral
Sentiment grouping
12 positive, 11 negative, 4 ambiguous
Inter-rater agreement
94% of examples have 2+ raters agreeing on a label
Filtered training set
43,410 examples
BERT baseline
Average F1 = .46
Select all that apply:

Configuration Fileconfig.yaml

This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.

yaml
# GoEmotions - Fine-Grained Emotion Classification
# Based on Demszky et al., ACL 2020
# Paper: https://aclanthology.org/2020.acl-main.372/
# Dataset: https://github.com/google-research/google-research/tree/master/goemotions
#
# Taxonomy: 27 emotions organized by sentiment:
# - Positive (12): admiration, amusement, approval, caring, desire, excitement,
#                  gratitude, joy, love, optimism, pride, relief
# - Negative (11): anger, annoyance, disappointment, disapproval, disgust,
#                  embarrassment, fear, grief, nervousness, remorse, sadness
# - Ambiguous (4): confusion, curiosity, realization, surprise
# - Plus: neutral
#
# Guidelines:
# - Select ALL emotions expressed in the text (multi-label)
# - Focus on emotion expressed by author, not evoked in reader
# - Related emotions differ by intensity (annoyance -> anger)

annotation_task_name: "GoEmotions: Fine-Grained Emotion Classification"
task_dir: "."

data_files:
  - sample-data.json
item_properties:
  id_key: "id"
  text_key: "text"

output_annotation_dir: "annotation_output/"
output_annotation_format: "json"

annotation_schemes:
  - annotation_type: multiselect
    name: emotions
    description: "Select ALL emotions expressed in this text (multiple selections allowed)"
    labels:
      # Positive emotions (12)
      - Admiration
      - Amusement
      - Approval
      - Caring
      - Desire
      - Excitement
      - Gratitude
      - Joy
      - Love
      - Optimism
      - Pride
      - Relief
      # Negative emotions (11)
      - Anger
      - Annoyance
      - Disappointment
      - Disapproval
      - Disgust
      - Embarrassment
      - Fear
      - Grief
      - Nervousness
      - Remorse
      - Sadness
      # Ambiguous emotions (4)
      - Confusion
      - Curiosity
      - Realization
      - Surprise

    keyboard_shortcuts:
      Admiration: "1"
      Amusement: "2"
      Approval: "3"
      Caring: "4"
      Joy: "5"
      Love: "6"
      Anger: "7"
      Sadness: "8"
      Fear: "9"
      Surprise: "0"

    tooltips:
      Admiration: "Finding something impressive or worthy of respect"
      Amusement: "Finding something funny or being entertained"
      Approval: "Having or expressing a favorable opinion"
      Caring: "Displaying kindness and concern for others"
      Desire: "A strong feeling of wanting something"
      Excitement: "Feeling very enthusiastic and eager"
      Gratitude: "Being thankful, ready to show appreciation"
      Joy: "A feeling of great pleasure and happiness"
      Love: "A strong positive feeling of affection"
      Optimism: "Hopefulness and confidence about the future"
      Pride: "Pleasure or satisfaction from achievements"
      Relief: "Reassurance and relaxation after anxiety"
      Anger: "A strong feeling of displeasure or hostility"
      Annoyance: "Slight irritation or frustration"
      Disappointment: "Sadness caused by unfulfilled expectations"
      Disapproval: "Possession or expression of an unfavorable opinion"
      Disgust: "A strong feeling of aversion or revulsion"
      Embarrassment: "Self-conscious awkwardness or shame"
      Fear: "Being afraid or worried about something"
      Grief: "Deep sorrow, especially caused by loss"
      Nervousness: "Easily agitated or worried"
      Remorse: "Deep regret or guilt for wrongdoing"
      Sadness: "Feeling or showing sorrow; unhappiness"
      Confusion: "Lack of understanding; uncertainty"
      Curiosity: "A strong desire to know or learn something"
      Realization: "Becoming fully aware of something"
      Surprise: "Feeling of astonishment or shock"

  - annotation_type: radio
    name: is_neutral
    description: "Is this text emotionally neutral?"
    labels:
      - "Contains emotion(s)"
      - "Neutral (no emotion)"
    keyboard_shortcuts:
      "Contains emotion(s)": "e"
      "Neutral (no emotion)": "n"

allow_all_users: true
instances_per_annotator: 200
annotation_per_instance: 3
allow_skip: true
skip_reason_required: false

Sample Datasample-data.json

json
[
  {
    "id": "goemotions_001",
    "text": "This is absolutely incredible! I can't believe how talented you are. Keep up the amazing work!"
  },
  {
    "id": "goemotions_002",
    "text": "Ugh, I've been waiting for 3 hours and still nothing. This is so frustrating."
  }
]

// ... and 10 more items

Try it live — no install

Boot the real Potato server in your browser (WebAssembly) and annotate with this exact config. Nothing leaves your machine.

▶ Run live in your browser

Get This Design

View on GitHub

Clone or download from the repository

Quick start:

git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/text/emotion-sentiment/goemotions
potato start config.yaml

Dataset & paper

Demszky et al., ACL 2020

Official dataset ↗Read the paper ↗

Citation (BibTeX)

bibtex
@inproceedings{demszky-etal-2020-goemotions,
    title = "{G}o{E}motions: A Dataset of Fine-Grained Emotions",
    author = "Demszky, Dorottya and Movshovitz-Attias, Dana and Ko, Jeongwoo and Cowen, Alan and Nemade, Gaurav and Ravi, Sujith",
    booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics",
    month = jul,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2020.acl-main.372",
    pages = "4040--4054"
}

Details

Annotation Types

multiselectradio

Domain

NLPSocial Media

Use Cases

Emotion DetectionSentiment AnalysisSocial Media Analysis

Tags

emotionmulti-labelredditfine-grainedgoemotionsacl2020

Found an issue or want to improve this design?

Open an Issue

Related Designs

AfriHate - Hate and Abusive Language for African Languages

Multilingual content-moderation annotation following the AfriHate scheme (Muhammad et al., NAACL 2025), a collection of hate speech and abusive language datasets for 15 African languages including Amharic, Hausa, Igbo, Yoruba, Swahili, Somali and isiZulu. Native speakers familiar with the regional culture assign each tweet a three-way label - hate, abusive, or neutral - and, when the tweet is hateful, mark the targeted attribute (e.g. ethnicity, religion, politics, gender). Sample items are mild, constructed illustrations that avoid real slurs or targeting of real groups; an English gloss is provided for reference only.

radiomultiselect

iSarcasmEval: Intended Sarcasm Detection

SemEval-2022 Task 6 (iSarcasmEval): detecting intended sarcasm in English and Arabic. Unlike prior datasets labeled by third parties, iSarcasmEval texts are labeled by their own authors for intended sarcasm. Annotators decide whether a text is sarcastic and, if so, which category of ironic speech it expresses.

radiomultiselect

Moral Foundations in Tweets

Classification of moral foundations in social media discourse, based on Moral Foundations Theory (Johnson & Goldwasser, ACL 2018). Annotators identify which moral dimensions are expressed and whether the tweet conveys moral sentiment.

multiselectradio