Persuasion Techniques in Memes
Detection and classification of persuasion techniques in memes, requiring annotators to identify rhetorical and propaganda techniques in meme text and highlight persuasive spans. Based on SemEval-2024 Task 4.
About this dataset
SemEval-2024 Task 4 asked systems to detect persuasion techniques in memes. It was organized by Dimitar Dimitrov, Firoj Alam, Maram Hasanain, Abul Hasnat, Fabrizio Silvestri, Preslav Nakov, and Giovanni Da San Martino, and published in the Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024). Given a meme's text and image, the goal is to identify which rhetorical and propaganda techniques it uses.
The task splits into three subtasks: deciding whether a meme uses any persuasion technique, labeling techniques in the meme's textual content alone, and labeling techniques across both the text and the image (a multimodal setting). It covered four languages, with surprise test sets in Bulgarian, North Macedonian, and Arabic added at evaluation time.
Techniques are drawn from a hierarchy of 22 persuasion techniques. The shared task drew 153 registered teams, of which 48 submitted results and 32 wrote system description papers.
The Potato config below reproduces this task with a multiselect scheme covering a nine-technique subset of the taxonomy, paired with a span scheme for highlighting the exact words that carry each technique. Each meme is shown with its text and an image description, so annotators can select the techniques present and mark the spans that trigger them.
- Persuasion techniques
- 22 (hierarchical)
- Languages
- 4 (English + Bulgarian, N. Macedonian, Arabic test)
- Subtasks
- 3
- Registered teams
- 153
- Teams submitting results
- 48
- System description papers
- 32
Configuration Fileconfig.yaml
This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.
# Persuasion Techniques in Memes
# Based on Dimitrov et al., SemEval 2024
# Paper: https://aclanthology.org/volumes/2024.semeval-1/
# Dataset: https://github.com/SemEval/semeval-2024-task4
#
# This task asks annotators to identify persuasion and propaganda techniques
# used in meme text, and to highlight the specific text spans where
# persuasive techniques are employed.
annotation_task_name: "Persuasion Techniques in Memes"
task_dir: "."
data_files:
- sample-data.json
item_properties:
id_key: "id"
text_key: "text"
output_annotation_dir: "annotation_output/"
output_annotation_format: "json"
port: 8000
server_name: localhost
annotation_schemes:
- annotation_type: multiselect
name: persuasion_techniques
description: "Select all persuasion techniques present in the meme."
labels:
- "Name Calling"
- "Loaded Language"
- "Appeal to Fear"
- "Bandwagon"
- "Exaggeration"
- "Black and White"
- "Thought-Terminating Cliche"
- "Smears"
- "Appeal to Authority"
tooltips:
"Name Calling": "Labeling the target with a negative term to discredit without evidence"
"Loaded Language": "Using emotionally charged words to influence the audience"
"Appeal to Fear": "Creating fear to persuade the audience to accept a conclusion"
"Bandwagon": "Suggesting everyone is doing it so you should too"
"Exaggeration": "Overstating or amplifying claims beyond reason"
"Black and White": "Presenting only two options when more exist"
"Thought-Terminating Cliche": "Using a common phrase to end critical thinking"
"Smears": "Attacking a person's character rather than their argument"
"Appeal to Authority": "Using an authority figure to support a claim without evidence"
- annotation_type: span
name: persuasion_spans
description: "Highlight the text spans that contain persuasion techniques."
labels:
- "Persuasion Span"
annotation_instructions: |
You will be shown the text from a meme along with a description of the image. Your task is to:
1. Read the meme text and image description carefully.
2. Identify all persuasion or propaganda techniques used.
3. Highlight the specific text spans where techniques are applied.
html_layout: |
<div style="padding: 15px; max-width: 800px; margin: auto;">
<div style="background: #f0f9ff; border: 1px solid #bae6fd; border-radius: 8px; padding: 16px; margin-bottom: 16px;">
<strong style="color: #0369a1;">Meme Text:</strong>
<p style="font-size: 16px; line-height: 1.7; margin: 8px 0 0 0;">{{text}}</p>
</div>
<div style="background: #fefce8; border: 1px solid #fde68a; border-radius: 8px; padding: 16px; margin-bottom: 16px;">
<strong style="color: #a16207;">Image Description:</strong>
<p style="font-size: 15px; line-height: 1.6; margin: 8px 0 0 0;">{{image_description}}</p>
</div>
<div style="background: #f0fdf4; border: 1px solid #bbf7d0; border-radius: 8px; padding: 12px;">
<strong style="color: #166534;">Language:</strong> <span>{{language}}</span>
</div>
</div>
allow_all_users: true
instances_per_annotator: 50
annotation_per_instance: 2
allow_skip: true
skip_reason_required: false
Sample Datasample-data.json
[
{
"id": "meme_001",
"text": "Only an absolute IDIOT would believe mainstream science. Wake up, sheeple! Every real expert agrees with us.",
"image_description": "A person with wide eyes looking shocked, with text overlaid on a dark background.",
"language": "English"
},
{
"id": "meme_002",
"text": "If we don't act NOW, our entire way of life will be DESTROYED forever. There is no middle ground.",
"image_description": "A dramatic image of a crumbling building with fire in the background.",
"language": "English"
}
]
// ... and 8 more itemsGet This Design
Clone or download from the repository
Quick start:
git clone https://github.com/davidjurgens/potato-showcase.git cd potato-showcase/semeval/2024/task04-persuasion-memes potato start config.yaml
Dataset & paper
Dimitrov et al., SemEval-2024
Citation (BibTeX)
@inproceedings{dimitrov-etal-2024-semeval,
title = "{S}em{E}val-2024 Task 4: Multilingual Detection of Persuasion Techniques in Memes",
author = "Dimitrov, Dimitar and Alam, Firoj and Hasanain, Maram and Hasnat, Abul and Silvestri, Fabrizio and Nakov, Preslav and Da San Martino, Giovanni",
booktitle = "Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)",
year = "2024",
pages = "2009--2026",
publisher = "Association for Computational Linguistics"
}Details
Annotation Types
Domain
Use Cases
Tags
Found an issue or want to improve this design?
Open an IssueRelated Designs
Detecting Persuasion Techniques in News
Identification of propaganda and persuasion techniques in news articles through both multi-label classification and span-level detection. Based on SemEval-2023 Task 3 (Piskorski et al.).
Food Hazard Detection
Food safety hazard detection task requiring annotators to identify hazards, products, and risk levels in food incident reports, and classify the type of contamination. Based on SemEval-2025 Task 9.
Multilingual Narrative Extraction
Multilingual narrative extraction task requiring annotators to identify narrative elements such as events, actors, and causal relations in news texts, and classify the narrative themes. Based on SemEval-2025 Task 10.