Skip to content
Showcase/Persuasion Techniques in Memes
advancedtext

Persuasion Techniques in Memes

Detection and classification of persuasion techniques in memes, requiring annotators to identify rhetorical and propaganda techniques in meme text and highlight persuasive spans. Based on SemEval-2024 Task 4.

About this dataset

SemEval-2024 Task 4 asked systems to detect persuasion techniques in memes. It was organized by Dimitar Dimitrov, Firoj Alam, Maram Hasanain, Abul Hasnat, Fabrizio Silvestri, Preslav Nakov, and Giovanni Da San Martino, and published in the Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024). Given a meme's text and image, the goal is to identify which rhetorical and propaganda techniques it uses.

The task splits into three subtasks: deciding whether a meme uses any persuasion technique, labeling techniques in the meme's textual content alone, and labeling techniques across both the text and the image (a multimodal setting). It covered four languages, with surprise test sets in Bulgarian, North Macedonian, and Arabic added at evaluation time.

Techniques are drawn from a hierarchy of 22 persuasion techniques. The shared task drew 153 registered teams, of which 48 submitted results and 32 wrote system description papers.

The Potato config below reproduces this task with a multiselect scheme covering a nine-technique subset of the taxonomy, paired with a span scheme for highlighting the exact words that carry each technique. Each meme is shown with its text and an image description, so annotators can select the techniques present and mark the spans that trigger them.

Persuasion techniques
22 (hierarchical)
Languages
4 (English + Bulgarian, N. Macedonian, Arabic test)
Subtasks
3
Registered teams
153
Teams submitting results
48
System description papers
32
PERORGLOCPERORGLOCDATESelect text to annotate

Configuration Fileconfig.yaml

This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.

yaml
# Persuasion Techniques in Memes
# Based on Dimitrov et al., SemEval 2024
# Paper: https://aclanthology.org/volumes/2024.semeval-1/
# Dataset: https://github.com/SemEval/semeval-2024-task4
#
# This task asks annotators to identify persuasion and propaganda techniques
# used in meme text, and to highlight the specific text spans where
# persuasive techniques are employed.

annotation_task_name: "Persuasion Techniques in Memes"
task_dir: "."

data_files:
  - sample-data.json

item_properties:
  id_key: "id"
  text_key: "text"

output_annotation_dir: "annotation_output/"
output_annotation_format: "json"

port: 8000
server_name: localhost

annotation_schemes:
  - annotation_type: multiselect
    name: persuasion_techniques
    description: "Select all persuasion techniques present in the meme."
    labels:
      - "Name Calling"
      - "Loaded Language"
      - "Appeal to Fear"
      - "Bandwagon"
      - "Exaggeration"
      - "Black and White"
      - "Thought-Terminating Cliche"
      - "Smears"
      - "Appeal to Authority"
    tooltips:
      "Name Calling": "Labeling the target with a negative term to discredit without evidence"
      "Loaded Language": "Using emotionally charged words to influence the audience"
      "Appeal to Fear": "Creating fear to persuade the audience to accept a conclusion"
      "Bandwagon": "Suggesting everyone is doing it so you should too"
      "Exaggeration": "Overstating or amplifying claims beyond reason"
      "Black and White": "Presenting only two options when more exist"
      "Thought-Terminating Cliche": "Using a common phrase to end critical thinking"
      "Smears": "Attacking a person's character rather than their argument"
      "Appeal to Authority": "Using an authority figure to support a claim without evidence"

  - annotation_type: span
    name: persuasion_spans
    description: "Highlight the text spans that contain persuasion techniques."
    labels:
      - "Persuasion Span"

annotation_instructions: |
  You will be shown the text from a meme along with a description of the image. Your task is to:
  1. Read the meme text and image description carefully.
  2. Identify all persuasion or propaganda techniques used.
  3. Highlight the specific text spans where techniques are applied.

html_layout: |
  <div style="padding: 15px; max-width: 800px; margin: auto;">
    <div style="background: #f0f9ff; border: 1px solid #bae6fd; border-radius: 8px; padding: 16px; margin-bottom: 16px;">
      <strong style="color: #0369a1;">Meme Text:</strong>
      <p style="font-size: 16px; line-height: 1.7; margin: 8px 0 0 0;">{{text}}</p>
    </div>
    <div style="background: #fefce8; border: 1px solid #fde68a; border-radius: 8px; padding: 16px; margin-bottom: 16px;">
      <strong style="color: #a16207;">Image Description:</strong>
      <p style="font-size: 15px; line-height: 1.6; margin: 8px 0 0 0;">{{image_description}}</p>
    </div>
    <div style="background: #f0fdf4; border: 1px solid #bbf7d0; border-radius: 8px; padding: 12px;">
      <strong style="color: #166534;">Language:</strong> <span>{{language}}</span>
    </div>
  </div>

allow_all_users: true
instances_per_annotator: 50
annotation_per_instance: 2
allow_skip: true
skip_reason_required: false

Sample Datasample-data.json

json
[
  {
    "id": "meme_001",
    "text": "Only an absolute IDIOT would believe mainstream science. Wake up, sheeple! Every real expert agrees with us.",
    "image_description": "A person with wide eyes looking shocked, with text overlaid on a dark background.",
    "language": "English"
  },
  {
    "id": "meme_002",
    "text": "If we don't act NOW, our entire way of life will be DESTROYED forever. There is no middle ground.",
    "image_description": "A dramatic image of a crumbling building with fire in the background.",
    "language": "English"
  }
]

// ... and 8 more items

Get This Design

View on GitHub

Clone or download from the repository

Quick start:

git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/semeval/2024/task04-persuasion-memes
potato start config.yaml

Dataset & paper

Dimitrov et al., SemEval-2024

Citation (BibTeX)

bibtex
@inproceedings{dimitrov-etal-2024-semeval,
    title = "{S}em{E}val-2024 Task 4: Multilingual Detection of Persuasion Techniques in Memes",
    author = "Dimitrov, Dimitar and Alam, Firoj and Hasanain, Maram and Hasnat, Abul and Silvestri, Fabrizio and Nakov, Preslav and Da San Martino, Giovanni",
    booktitle = "Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)",
    year = "2024",
    pages = "2009--2026",
    publisher = "Association for Computational Linguistics"
}

Details

Annotation Types

multiselectspan

Domain

SemEvalNLPPersuasion DetectionMultimodal

Use Cases

Propaganda DetectionPersuasion AnalysisMedia Literacy

Tags

semevalsemeval-2024shared-taskpersuasionmemespropagandamultimodal

Found an issue or want to improve this design?

Open an Issue