POLAR: Detecting Multilingual, Multicultural and Multievent Online Polarization
SemEval-2026 Task 9 (POLAR): detecting online polarization across 22 languages, many cultures, and multiple real-world events. Annotators judge whether a social-media post is polarized and, if so, identify the type of polarization and how it is rhetorically manifested (e.g. stereotyping, vilification, dehumanization, intolerance). Polarization encompasses stereotyping, vilification, dehumanization, and intolerance directed at social groups.
Configuration Fileconfig.yaml
This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.
# POLAR: Detecting Multilingual, Multicultural and Multievent Online Polarization
# Based on Naseem et al., SemEval-2026 Task 9
# Paper: https://aclanthology.org/2026.semeval-1.453/
# Task page: https://polar-semeval.github.io/
#
# Given a social-media post (one of 22 languages), judge whether it is
# polarized. The shared task has three subtasks:
# - Subtask 1: binary polarization detection (polarized / not)
# - Subtask 2: polarization type (the social dimension involved)
# - Subtask 3: manifestation categories (how polarization is expressed)
# The organizers characterize polarization as encompassing stereotyping,
# vilification, dehumanization, and intolerance toward social groups; those
# form the manifestation labels below. This showcase collects the binary
# label, the manifestation(s), and the polarization type as free text
# (the official type inventory is defined by the task).
annotation_task_name: "POLAR - Online Polarization"
task_dir: "."
data_files:
- sample-data.json
item_properties:
id_key: "id"
text_key: "text"
output_annotation_dir: "annotation_output/"
output_annotation_format: "json"
port: 8000
server_name: localhost
annotation_schemes:
- annotation_type: radio
name: polarized
description: "Is this post polarized?"
labels:
- "Polarized"
- "Not polarized"
keyboard_shortcuts:
"Polarized": "1"
"Not polarized": "2"
- annotation_type: multiselect
name: manifestation
description: "If polarized, how is it manifested? (Select all that apply)"
labels:
- "Stereotyping"
- "Vilification"
- "Dehumanization"
- "Intolerance"
tooltips:
"Stereotyping": "Attributing fixed, oversimplified traits to a group"
"Vilification": "Portraying a group as evil, blameworthy, or threatening"
"Dehumanization": "Denying a group's humanity (e.g. comparing to animals/objects)"
"Intolerance": "Rejecting a group's right to exist, speak, or participate"
- annotation_type: text
name: polarization_type
description: "If polarized, what is the target dimension/type (e.g. political, religious, ethnic, gender)? - optional"
textarea: false
required: false
annotation_instructions: |
Read the social-media post (an English gloss is provided for reference).
First decide whether it is polarized. If it is, select how the
polarization is manifested (stereotyping, vilification, dehumanization,
intolerance) and note the target dimension (e.g. political, religious,
ethnic, gender).
html_layout: |
<div style="padding: 15px; max-width: 800px; margin: auto;">
<div style="background: #fff1f2; border: 1px solid #fecdd3; border-radius: 8px; padding: 8px 12px; margin-bottom: 10px;">
<strong style="color: #9f1239;">Language:</strong> <span>{{language}}</span>
</div>
<div style="background: #fef2f2; border: 1px solid #fecaca; border-radius: 8px; padding: 16px; margin-bottom: 8px;">
<strong style="color: #991b1b;">Post:</strong>
<p style="font-size: 16px; line-height: 1.7; margin: 8px 0 0 0;">{{text}}</p>
</div>
<div style="color: #6b7280; font-size: 13px; font-style: italic;">English gloss (reference): {{gloss}}</div>
</div>
allow_all_users: true
instances_per_annotator: 50
annotation_per_instance: 3
allow_skip: true
Sample Datasample-data.json
[
{
"id": "pol_001",
"language": "English",
"text": "The city council approved the new bike lanes after months of public consultation.",
"gloss": "The city council approved the new bike lanes after months of public consultation."
},
{
"id": "pol_002",
"language": "English",
"text": "Those people are all the same, lazy and dishonest, and they'll ruin this country if we let them.",
"gloss": "Those people are all the same, lazy and dishonest, and they'll ruin this country if we let them."
}
]
// ... and 8 more itemsGet This Design
Clone or download from the repository
Quick start:
git clone https://github.com/davidjurgens/potato-showcase.git cd potato-showcase/semeval/2026/task09-polar-polarization potato start config.yaml
Dataset & paper
Naseem et al., SemEval 2026 (Task 9)
Citation (BibTeX)
@inproceedings{naseem-etal-2026-semeval,
title = "{S}em{E}val-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization",
author = "Naseem, Usman and Geislinger, Robert and Ren, Ada and Kohail, Sarah and Garrido Veliz, Rudy and Muhammad, Shamsuddeen Hassan and Yimam, Seid Muhie and others",
booktitle = "Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026)",
year = "2026",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2026.semeval-1.453",
pages = "3779--3796"
}Details
Annotation Types
Domain
Use Cases
Tags
Found an issue or want to improve this design?
Open an IssueRelated Designs
iSarcasmEval: Intended Sarcasm Detection
SemEval-2022 Task 6 (iSarcasmEval): detecting intended sarcasm in English and Arabic. Unlike prior datasets labeled by third parties, iSarcasmEval texts are labeled by their own authors for intended sarcasm. Annotators decide whether a text is sarcastic and, if so, which category of ironic speech it expresses.
Media Frames Corpus: News Framing Annotation
The Media Frames Corpus (Card et al., ACL 2015) labels U.S. news articles on immigration, smoking, and same-sex marriage with 15 general framing dimensions. This Potato config reproduces that frame-coding task.
FLUTE: Figurative Language Understanding through Textual Explanations
Figurative language understanding via NLI. Annotators classify figurative sentences (sarcasm, simile, metaphor, idiom) and provide textual explanations of the figurative meaning. The task combines natural language inference with fine-grained figurative language type classification.