Skip to content
Showcase/HaHackathon - Detecting and Rating Humor and Offense
beginnersurvey

HaHackathon - Detecting and Rating Humor and Offense

Classify whether a text is humorous and rate its offensiveness on a Likert scale, based on SemEval-2021 Task 7 (Meaney et al.). The task combines binary humor detection with graded offense rating to study the relationship between humor and offensiveness.

Q1: Rate your experience12345Q2: Primary use case?ResearchIndustryEducationQ3: Additional feedback

Fichier de configurationconfig.yaml

# HaHackathon - Detecting and Rating Humor and Offense
# Based on Meaney et al., SemEval 2021
# Paper: https://aclanthology.org/2021.semeval-1.9/
# Dataset: https://competitions.codalab.org/competitions/27446
#
# Annotators first decide whether a text is funny, then rate how
# offensive it is on a 5-point scale from "Not Offensive" to "Very Offensive".

annotation_task_name: "HaHackathon - Detecting and Rating Humor and Offense"
task_dir: "."

data_files:
  - sample-data.json

item_properties:
  id_key: "id"
  text_key: "text"

output_annotation_dir: "annotation_output/"
output_annotation_format: "json"

port: 8000
server_name: localhost

annotation_schemes:
  - annotation_type: radio
    name: humor_judgment
    description: "Is this text funny?"
    labels:
      - "Funny"
      - "Not Funny"
    keyboard_shortcuts:
      "Funny": "1"
      "Not Funny": "2"
    tooltips:
      "Funny": "The text is intended to be humorous and succeeds in being funny"
      "Not Funny": "The text is not funny or the humor attempt fails"

  - annotation_type: likert
    name: offense_rating
    description: "Rate how offensive this text is."
    min_label: "Not Offensive"
    max_label: "Very Offensive"
    size: 5

annotation_instructions: |
  You will see a short text that may or may not be humorous. Your task is to:
  1. Read the text carefully.
  2. Decide whether the text is funny or not funny.
  3. Rate how offensive the text is on a scale from "Not Offensive" to "Very Offensive".

  Note that humor and offense are independent dimensions - a text can be funny
  and offensive, funny and inoffensive, or not funny but still offensive.

html_layout: |
  <div style="padding: 15px; max-width: 800px; margin: auto;">
    <div style="background: #f0f9ff; border: 1px solid #bae6fd; border-radius: 8px; padding: 16px; margin-bottom: 16px;">
      <strong style="color: #0369a1;">Text:</strong>
      <p style="font-size: 16px; line-height: 1.7; margin: 8px 0 0 0;">{{text}}</p>
    </div>
  </div>

allow_all_users: true
instances_per_annotator: 50
annotation_per_instance: 2
allow_skip: true
skip_reason_required: false

Données d'exemplesample-data.json

[
  {
    "id": "humor_001",
    "text": "I told my wife she was drawing her eyebrows too high. She looked surprised."
  },
  {
    "id": "humor_002",
    "text": "The weather forecast calls for rain tomorrow and continued disappointment throughout the week."
  }
]

// ... and 8 more items

Obtenir ce design

View on GitHub

Clone or download from the repository

Démarrage rapide :

git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/semeval/2021/task07-hahackathon-humor
potato start config.yaml

Détails

Types d'annotation

radiolikert

Domaine

NLPSemEval

Cas d'utilisation

Humor DetectionOffense DetectionContent Moderation

Étiquettes

semevalsemeval-2021shared-taskhumoroffenseclassification

Vous avez trouvé un problème ou souhaitez améliorer ce design ?

Ouvrir un ticket

Designs associés

Assessing Humor in Edited News Headlines

Rate the funniness of edited news headlines and compare humor between original and edited versions, based on SemEval-2020 Task 7 (Hossain et al.). Headlines are minimally edited by replacing a single word to create humorous effect.

likertradio

Fine-Grained Sentiment Analysis on Financial Microblogs and News

Graded sentiment analysis of financial text with topic classification, rating market sentiment from very bearish to very bullish on a 7-point scale. Based on SemEval-2017 Task 5.

likertradio

Unlearning Sensitive Content from LLMs

Evaluation of whether language models have successfully unlearned sensitive content, requiring annotators to assess model outputs for residual sensitive information leakage. Based on SemEval-2025 Task 4.

radiotext