Multilingual Semantic Role Labeling
Annotate predicate-argument structure in multilingual text by marking predicates, labeling their argument spans with semantic roles, and classifying voice type. Mirrors the topic of SemEval-2021 Task 3 (Span- and Dependency-based Multilingual and Cross-lingual Semantic Role Labeling); grounded in the UniteD-SRL dataset (Tripodi, Conia & Navigli, Findings of EMNLP 2021).
Configuration Fileconfig.yaml
This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.
# Multilingual Semantic Role Labeling
# Topic of SemEval-2021 Task 3 (Span- and Dependency-based Multilingual
# and Cross-lingual SRL); grounded in the UniteD-SRL dataset.
# Based on Tripodi, Conia & Navigli, Findings of EMNLP 2021
# Paper: https://aclanthology.org/2021.findings-emnlp.197/
# Dataset: https://github.com/SapienzaNLP/united-srl
#
# Annotators identify predicate-argument structures in sentences by
# marking spans for semantic roles (Predicate, Agent, Patient, etc.)
# and classifying the voice type of the main predicate.
annotation_task_name: "Multilingual Semantic Role Labeling"
task_dir: "."
data_files:
- sample-data.json
item_properties:
id_key: "id"
text_key: "text"
output_annotation_dir: "annotation_output/"
output_annotation_format: "json"
port: 8000
server_name: localhost
annotation_schemes:
- annotation_type: span
name: semantic_roles
description: "Highlight text spans and assign semantic role labels."
labels:
- "Predicate"
- "Agent"
- "Patient"
- "Theme"
- "Location"
- "Time"
- "Instrument"
- "Beneficiary"
- annotation_type: radio
name: voice_type
description: "What is the voice type of the main predicate?"
labels:
- "Active"
- "Passive"
- "Middle"
keyboard_shortcuts:
"Active": "1"
"Passive": "2"
"Middle": "3"
tooltips:
"Active": "The subject performs the action (e.g., 'The cat chased the mouse')"
"Passive": "The subject receives the action (e.g., 'The mouse was chased by the cat')"
"Middle": "The subject is both agent and patient (e.g., 'The door opened')"
annotation_instructions: |
You will see a sentence in a given language. Your tasks:
1. Use span annotation to mark the predicate and its arguments:
- Predicate: The main verb or action
- Agent: Who/what performs the action
- Patient: Who/what is affected by the action
- Theme: What is being moved or described
- Location, Time, Instrument, Beneficiary as applicable
2. Classify the voice type of the main predicate as Active, Passive, or Middle.
Not every role will be present in every sentence. Mark only the roles you find.
html_layout: |
<div style="padding: 15px; max-width: 800px; margin: auto;">
<div style="background: #f0f9ff; border: 1px solid #bae6fd; border-radius: 8px; padding: 16px; margin-bottom: 16px;">
<strong style="color: #0369a1;">Sentence:</strong>
<p style="font-size: 16px; line-height: 1.7; margin: 8px 0 0 0;">{{text}}</p>
</div>
<div style="background: #f8fafc; border: 1px solid #e2e8f0; border-radius: 8px; padding: 12px; margin-bottom: 16px;">
<strong style="color: #475569;">Language:</strong>
<span style="font-size: 15px;">{{language}}</span>
</div>
</div>
allow_all_users: true
instances_per_annotator: 50
annotation_per_instance: 2
allow_skip: true
skip_reason_required: false
Sample Datasample-data.json
[
{
"id": "srl_001",
"text": "The researcher published the results of the experiment in a prestigious journal last month.",
"language": "English"
},
{
"id": "srl_002",
"text": "The bridge was constructed by a team of engineers using reinforced concrete.",
"language": "English"
}
]
// ... and 8 more itemsGet This Design
Clone or download from the repository
Quick start:
git clone https://github.com/davidjurgens/potato-showcase.git cd potato-showcase/semeval/2021/task03-multilingual-srl potato start config.yaml
Dataset & paper
Tripodi, Conia & Navigli, Findings of EMNLP 2021 (UniteD-SRL)
Citation (BibTeX)
@inproceedings{tripodi-etal-2021-united-srl,
title = "{U}nite{D}-{SRL}: A Unified Dataset for Span- and Dependency-Based Multilingual and Cross-Lingual {S}emantic {R}ole {L}abeling",
author = "Tripodi, Rocco and Conia, Simone and Navigli, Roberto",
booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2021",
month = nov,
year = "2021",
address = "Punta Cana, Dominican Republic",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2021.findings-emnlp.197/",
doi = "10.18653/v1/2021.findings-emnlp.197",
pages = "2293--2305"
}Details
Annotation Types
Domain
Use Cases
Tags
Found an issue or want to improve this design?
Open an IssueRelated Designs
Aspect-Based Sentiment Analysis
Identification of aspect terms in review text with sentiment polarity classification for each aspect. Based on SemEval-2016 Task 5 (ABSA).
Character Identification on Multiparty Dialogues
Identification and linking of character mentions in TV show dialogue, combining span annotation with entity resolution for the main cast of Friends. Based on SemEval-2018 Task 4.
Clickbait Spoiling
Classification and extraction of spoilers for clickbait posts, including spoiler type identification and span-level spoiler detection. Based on SemEval-2023 Task 5 (Fröbe et al.).