Skip to content
advancedtext

Multilingual Semantic Role Labeling

Annotate predicate-argument structure in multilingual text by marking predicates, labeling their argument spans with semantic roles, and classifying voice type. Mirrors the topic of SemEval-2021 Task 3 (Span- and Dependency-based Multilingual and Cross-lingual Semantic Role Labeling); grounded in the UniteD-SRL dataset (Tripodi, Conia & Navigli, Findings of EMNLP 2021).

PERORGLOCPERORGLOCDATESelect text to annotate

Configuration Fileconfig.yaml

This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.

yaml
# Multilingual Semantic Role Labeling
# Topic of SemEval-2021 Task 3 (Span- and Dependency-based Multilingual
# and Cross-lingual SRL); grounded in the UniteD-SRL dataset.
# Based on Tripodi, Conia & Navigli, Findings of EMNLP 2021
# Paper: https://aclanthology.org/2021.findings-emnlp.197/
# Dataset: https://github.com/SapienzaNLP/united-srl
#
# Annotators identify predicate-argument structures in sentences by
# marking spans for semantic roles (Predicate, Agent, Patient, etc.)
# and classifying the voice type of the main predicate.

annotation_task_name: "Multilingual Semantic Role Labeling"
task_dir: "."

data_files:
  - sample-data.json

item_properties:
  id_key: "id"
  text_key: "text"

output_annotation_dir: "annotation_output/"
output_annotation_format: "json"

port: 8000
server_name: localhost

annotation_schemes:
  - annotation_type: span
    name: semantic_roles
    description: "Highlight text spans and assign semantic role labels."
    labels:
      - "Predicate"
      - "Agent"
      - "Patient"
      - "Theme"
      - "Location"
      - "Time"
      - "Instrument"
      - "Beneficiary"

  - annotation_type: radio
    name: voice_type
    description: "What is the voice type of the main predicate?"
    labels:
      - "Active"
      - "Passive"
      - "Middle"
    keyboard_shortcuts:
      "Active": "1"
      "Passive": "2"
      "Middle": "3"
    tooltips:
      "Active": "The subject performs the action (e.g., 'The cat chased the mouse')"
      "Passive": "The subject receives the action (e.g., 'The mouse was chased by the cat')"
      "Middle": "The subject is both agent and patient (e.g., 'The door opened')"

annotation_instructions: |
  You will see a sentence in a given language. Your tasks:

  1. Use span annotation to mark the predicate and its arguments:
     - Predicate: The main verb or action
     - Agent: Who/what performs the action
     - Patient: Who/what is affected by the action
     - Theme: What is being moved or described
     - Location, Time, Instrument, Beneficiary as applicable

  2. Classify the voice type of the main predicate as Active, Passive, or Middle.

  Not every role will be present in every sentence. Mark only the roles you find.

html_layout: |
  <div style="padding: 15px; max-width: 800px; margin: auto;">
    <div style="background: #f0f9ff; border: 1px solid #bae6fd; border-radius: 8px; padding: 16px; margin-bottom: 16px;">
      <strong style="color: #0369a1;">Sentence:</strong>
      <p style="font-size: 16px; line-height: 1.7; margin: 8px 0 0 0;">{{text}}</p>
    </div>
    <div style="background: #f8fafc; border: 1px solid #e2e8f0; border-radius: 8px; padding: 12px; margin-bottom: 16px;">
      <strong style="color: #475569;">Language:</strong>
      <span style="font-size: 15px;">{{language}}</span>
    </div>
  </div>

allow_all_users: true
instances_per_annotator: 50
annotation_per_instance: 2
allow_skip: true
skip_reason_required: false

Sample Datasample-data.json

json
[
  {
    "id": "srl_001",
    "text": "The researcher published the results of the experiment in a prestigious journal last month.",
    "language": "English"
  },
  {
    "id": "srl_002",
    "text": "The bridge was constructed by a team of engineers using reinforced concrete.",
    "language": "English"
  }
]

// ... and 8 more items

Get This Design

View on GitHub

Clone or download from the repository

Quick start:

git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/semeval/2021/task03-multilingual-srl
potato start config.yaml

Dataset & paper

Tripodi, Conia & Navigli, Findings of EMNLP 2021 (UniteD-SRL)

Citation (BibTeX)

bibtex
@inproceedings{tripodi-etal-2021-united-srl,
    title = "{U}nite{D}-{SRL}: A Unified Dataset for Span- and Dependency-Based Multilingual and Cross-Lingual {S}emantic {R}ole {L}abeling",
    author = "Tripodi, Rocco  and Conia, Simone  and Navigli, Roberto",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2021",
    month = nov,
    year = "2021",
    address = "Punta Cana, Dominican Republic",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.findings-emnlp.197/",
    doi = "10.18653/v1/2021.findings-emnlp.197",
    pages = "2293--2305"
}

Details

Annotation Types

spanradio

Domain

NLPSemEval

Use Cases

Semantic Role LabelingCross-lingual NLP

Tags

semevalsemeval-2021shared-tasksemantic-rolessrlmultilingual

Found an issue or want to improve this design?

Open an Issue