Skip to content
intermediatetext

MultiCoNER II: Multilingual Complex Named Entity Recognition

Complex and ambiguous named entity recognition across 12 languages. Annotators identify fine-grained entity types including creative works, groups, medical terms, and complex entities that require world knowledge. Based on the SemEval-2023 Task 2 shared task for multilingual complex NER.

Get This Design

View on GitHub(opens in a new tab)

Clone or download from the repository

Quick start:

bash
git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/text/domain-specific/multiconerii-complex-ner
potato start config.yaml

Configuration File

This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.

yaml
# MultiCoNER II: Multilingual Complex Named Entity Recognition
# Based on SemEval-2023 Task 2 (Fetahu et al., SemEval@ACL 2023)
# Paper: https://aclanthology.org/2023.semeval-1.310/
# Dataset: https://multiconer.github.io/dataset
#
# Task: Fine-grained NER across 12 languages with complex entities
# Annotators identify named entities from a fine-grained taxonomy
# that includes creative works, groups, medical terms, and more.
#
# Key Challenges:
# - Complex entities requiring world knowledge (e.g., song titles)
# - Ambiguous entities that could belong to multiple types
# - Multilingual text with code-switching
# - Low-context short sentences from search queries and social media
#
# Entity Types (6 coarse groups; each spans fine-grained subtypes):
# - Person: Scientist, Artist, Athlete, Politician, Cleric, SportsManager, OtherPER
# - Location: Facility, HumanSettlement, Station, OtherLOC
# - Group: MusicalGRP, PublicCORP, PrivateCORP, AerospaceManufacturer,
#          SportsGRP, CarManufacturer, ORG (companies/institutions)
# - CreativeWork: VisualWork, MusicalWork, WrittenWork, ArtWork, Software
# - Product: Clothing, Vehicle, Food, Drink, OtherPROD
# - Medical: Medication/Vaccine, MedicalProcedure, AnatomicalStructure, Symptom, Disease

annotation_task_name: "MultiCoNER II: Complex Named Entity Recognition"
task_dir: "."

data_files:
  - sample-data.json
item_properties:
  id_key: "id"
  text_key: "text"

output_annotation_dir: "annotation_output/"
output_annotation_format: "json"

annotation_schemes:
  - annotation_type: span
    name: entities
    description: "Highlight all named entities and assign their fine-grained type"
    labels:
      - "Person"
      - "Location"
      - "Group"
      - "CreativeWork"
      - "Product"
      - "Medical"
    label_colors:
      "Person": "#3b82f6"
      "Location": "#22c55e"
      "Group": "#f59e0b"
      "CreativeWork": "#ec4899"
      "Product": "#06b6d4"
      "Medical": "#ef4444"
    tooltips:
      "Person": "People: scientists, artists, athletes, politicians, clerics, sports managers, others (e.g., Albert Einstein, Lamine Yamal)"
      "Location": "Facilities, human settlements, stations, other locations (e.g., Tokyo, Elbphilharmonie, Ohio State Reformatory)"
      "Group": "Musical groups, public/private corporations, sports groups, aerospace/car manufacturers, and other organizations (e.g., Daft Punk, FC Barcelona, Sony, WHO)"
      "CreativeWork": "Visual works, musical works, written works, artworks, software (e.g., The Shawshank Redemption, PlayStation 5 Pro games)"
      "Product": "Clothing, vehicles, food, drink, other products (e.g., iPhone, Boeing 747, Snapdragon 8 Gen 3)"
      "Medical": "Medications/vaccines, medical procedures, anatomical structures, symptoms, diseases (e.g., COVID-19, remdesivir, rVSV-ZEBOV)"
    allow_overlapping: false

html_layout: |
  <div style="margin-bottom: 10px; padding: 8px; background: #f0f4ff; border-radius: 6px;">
    <strong>Language:</strong> {{language}}
  </div>
  <div style="padding: 10px; border: 1px solid #ddd; border-radius: 6px; line-height: 1.8; font-size: 16px;">
    {{text}}
  </div>

allow_all_users: true
instances_per_annotator: 100
annotation_per_instance: 2
allow_skip: true
skip_reason_required: false

Sample Data

json
[
  {
    "id": "mconer_001",
    "text": "The Shawshank Redemption, directed by Frank Darabont, was filmed at the Ohio State Reformatory in Mansfield.",
    "language": "English"
  },
  {
    "id": "mconer_002",
    "text": "Le groupe Daft Punk a annonce sa separation apres 28 ans de carriere, laissant des fans du monde entier en deuil.",
    "language": "French"
  }
]

// ... and 8 more items

Dataset & paper

Fetahu et al., SemEval@ACL 2023

Citation (BibTeX)

bibtex
@inproceedings{fetahu-etal-2023-semeval,
    title = "{S}em{E}val-2023 Task 2: Fine-Grained Multilingual Named Entity Recognition ({M}ulti{C}o{NER} 2)",
    author = "Fetahu, Besnik  and Kar, Sudipta  and Chen, Zhiyu  and Rokhlenko, Oleg  and Malmasi, Shervin",
    booktitle = "Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.semeval-1.310",
    pages = "2247--2265"
}

Details

Annotation Types

span

Domain

NLPNamed Entity RecognitionMultilingual

Use Cases

Named Entity RecognitionMultilingual NLPKnowledge Base Population

Tags

nermultilingualmulticonersemeval2023complex-entitiesfine-grained-ner

Found an issue or want to improve this design?

Open an Issue(opens in a new tab)