Counting Events and Participants in News

Counting the number of events or incidents described in news reports and providing event descriptions. Based on SemEval-2018 Task 5 (Counting Events and Participants).

Configuration Fileconfig.yaml

This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.

yaml

# Counting Events and Participants in News
# Based on Postma et al., SemEval 2018
# Paper: https://aclanthology.org/S18-1009/
# Dataset: http://www.newsreader-project.eu/results/data/wikinews/
#
# This task asks annotators to count the number of distinct events or
# incidents described in a news article and provide a brief description
# of each event.
#
# Annotators enter a number for the event count and provide a textual
# description summarizing the distinct events found.

annotation_task_name: "Counting Events and Participants in News"
task_dir: "."

data_files:
  - sample-data.json

item_properties:
  id_key: "id"
  text_key: "text"

output_annotation_dir: "annotation_output/"
output_annotation_format: "json"

port: 8000
server_name: localhost

annotation_schemes:
  - annotation_type: number
    name: event_count
    description: "How many distinct events or incidents are described in this text?"

  - annotation_type: text
    name: event_description
    description: "Briefly describe each distinct event mentioned in the text."

annotation_instructions: |
  You will be shown a news article along with its event type category.
  Your task is to:
  1. Count the number of distinct events or incidents described in the text.
  2. Provide a brief description of each event identified.
  Note: Related follow-up actions count as separate events if they are distinct incidents.

html_layout: |
  <div style="padding: 15px; max-width: 800px; margin: auto;">
    <div style="background: #fefce8; border: 1px solid #fde68a; border-radius: 8px; padding: 12px; margin-bottom: 12px;">
      <strong style="color: #a16207;">Event Type:</strong>
      <span style="font-size: 15px;">{{event_type}}</span>
    </div>
    <div style="background: #f0f9ff; border: 1px solid #bae6fd; border-radius: 8px; padding: 16px; margin-bottom: 16px;">
      <strong style="color: #0369a1;">News Article:</strong>
      <p style="font-size: 16px; line-height: 1.7; margin: 8px 0 0 0;">{{text}}</p>
    </div>
  </div>

allow_all_users: true
instances_per_annotator: 50
annotation_per_instance: 2
allow_skip: true
skip_reason_required: false

Sample Datasample-data.json

json

[
  {
    "id": "eventcount_001",
    "text": "A magnitude 6.2 earthquake struck central Italy on Wednesday, causing widespread damage to buildings in Amatrice. Rescue teams recovered 12 survivors from the rubble. A 4.5 magnitude aftershock hit the same region the following day.",
    "event_type": "Natural Disaster"
  },
  {
    "id": "eventcount_002",
    "text": "Police arrested three suspects in connection with the robbery of a downtown jewelry store. The heist, which occurred last Tuesday, resulted in the theft of over $500,000 in merchandise.",
    "event_type": "Crime"
  }
]

// ... and 8 more items

Get This Design

View on GitHub

Clone or download from the repository

Quick start:

git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/semeval/2018/task05-event-counting
potato start config.yaml

Dataset & paper

Postma et al., SemEval 2018

Official dataset ↗Read the paper ↗

Citation (BibTeX)

bibtex

@inproceedings{postma-etal-2018-semeval,
    title = "{S}em{E}val-2018 Task 5: Counting Events and Participants in the Long Tail",
    author = "Postma, Marten and Ilievski, Filip and Vossen, Piek",
    booktitle = "Proceedings of the 12th International Workshop on Semantic Evaluation",
    year = "2018",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/S18-1009"
}

Details

Annotation Types

numbertext

Domain

SemEvalNLPEvent DetectionInformation Extraction

Use Cases

Event CountingNews AnalysisInformation Extraction

Related Designs

NumEval - Numeral-Aware Language Understanding

Numeral-aware language understanding task requiring annotators to predict numerical values from text, classify numeral types, and provide explanations. Based on SemEval-2024 Task 7 (NumEval).

numberradio

Parsing Time Normalizations

Parsing time expressions in text using the SCATE (Semantically Compositional Annotation of Time Expressions) schema, marking typed time entities and operators (e.g., Year, Month-Of-Year, Day-Of-Month, Last, Next, This, Between) that compose into interval semantics. Based on SemEval-2018 Task 6.

spantext

Argument Reasoning in Civil Procedure

Legal argument reasoning task requiring annotators to answer multiple-choice questions about civil procedure by selecting the best answer and providing legal reasoning. Based on SemEval-2024 Task 5.

radiotext