Skip to content
Showcase/Question Answering
intermediatetext

Question Answering

Annotate answer spans in text passages for reading comprehension tasks.

PERORGLOCPERORGLOCDATESelect text to annotate

Configuration Fileconfig.yaml

annotation_task_name: "Question Answering"

task_description: "Select the span in the passage that answers the question."
task_dir: "."
port: 8000

data_files:
  - "sample-data.json"

item_properties:
  id_key: id
  text_key: passage
  context_key: question

annotation_schemes:
  - annotation_type: span
    name: answer
    description: "Highlight the answer span in the passage"
    labels:
      - name: "Answer"
        color: "#22c55e"
    allow_overlapping: false
    required: true

  - annotation_type: radio
    name: answerable
    description: "Is the question answerable from the passage?"
    labels:
      - "Yes"
      - "No"
    required: true

output_annotation_dir: "output/"
output_annotation_format: "json"

Sample Datasample-data.json

[
  {
    "id": "1",
    "passage": "The Eiffel Tower is a wrought-iron lattice tower on the Champ de Mars in Paris, France. It was constructed from 1887 to 1889 as the centerpiece of the 1889 World's Fair. The tower is 330 metres tall and was the tallest man-made structure in the world until 1930.",
    "question": "When was the Eiffel Tower built?"
  },
  {
    "id": "2",
    "passage": "Python is a high-level programming language created by Guido van Rossum and first released in 1991. It emphasizes code readability with its use of significant indentation.",
    "question": "Who created Python?"
  }
]

Get This Design

View on GitHub

Clone or download from the repository

Quick start:

git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/evaluation/question-answering
potato start config.yaml

Details

Annotation Types

spanradio

Domain

NLP

Use Cases

Question AnsweringReading Comprehension

Tags

qaspanreading-comprehension

Found an issue or want to improve this design?

Open an Issue

Related Designs

Aspect-Based Sentiment Analysis

Identification of aspect terms in review text with sentiment polarity classification for each aspect. Based on SemEval-2016 Task 5 (ABSA).

spanradio

Causal Medical Claim Detection and PICO Extraction

Detection of causal claims in medical texts and extraction of PICO (Population, Intervention, Comparator, Outcome) elements. Based on SemEval-2023 Task 8 (Khetan et al.).

spanradio

Character Identification on Multiparty Dialogues

Identification and linking of character mentions in TV show dialogue, combining span annotation with entity resolution for the main cast of Friends. Based on SemEval-2018 Task 4.

spanradio