Skip to content
intermediatetext

Question Answering

Annotate answer spans in text passages for reading comprehension tasks.

PERORGLOCPERORGLOCDATESelect text to annotate

Configuration Fileconfig.yaml

yaml
annotation_task_name: "Question Answering"

task_description: "Select the span in the passage that answers the question."
task_dir: "."
port: 8000

data_files:
  - "sample-data.json"

item_properties:
  id_key: id
  text_key: passage
  context_key: question

annotation_schemes:
  - annotation_type: span
    name: answer
    description: "Highlight the answer span in the passage"
    labels:
      - name: "Answer"
        color: "#22c55e"
    allow_overlapping: false
    required: true

  - annotation_type: radio
    name: answerable
    description: "Is the question answerable from the passage?"
    labels:
      - "Yes"
      - "No"
    required: true

output_annotation_dir: "output/"
output_annotation_format: "json"

Sample Datasample-data.json

json
[
  {
    "id": "1",
    "passage": "The Eiffel Tower is a wrought-iron lattice tower on the Champ de Mars in Paris, France. It was constructed from 1887 to 1889 as the centerpiece of the 1889 World's Fair. The tower is 330 metres tall and was the tallest man-made structure in the world until 1930.",
    "question": "When was the Eiffel Tower built?"
  },
  {
    "id": "2",
    "passage": "Python is a high-level programming language created by Guido van Rossum and first released in 1991. It emphasizes code readability with its use of significant indentation.",
    "question": "Who created Python?"
  }
]

Try it live — no install

Boot the real Potato server in your browser (WebAssembly) and annotate with this exact config. Nothing leaves your machine.

▶ Run live in your browser

Get This Design

View on GitHub

Clone or download from the repository

Quick start:

git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/evaluation/question-answering
potato start config.yaml

Details

Annotation Types

spanradio

Domain

NLP

Use Cases

Question AnsweringReading Comprehension

Tags

qaspanreading-comprehension

Found an issue or want to improve this design?

Open an Issue

Related Designs

Aspect-Based Sentiment Analysis

Identification of aspect terms in review text with sentiment polarity classification for each aspect. Based on SemEval-2016 Task 5 (ABSA).

spanradio

Character Identification on Multiparty Dialogues

Identification and linking of character mentions in TV show dialogue, combining span annotation with entity resolution for the main cast of Friends. Based on SemEval-2018 Task 4.

spanradio

Check-COVID: Fact-Checking COVID-19 News Claims

Fact-checking COVID-19 news claims. Annotators verify claims against evidence, identify supporting/refuting spans, and provide verdicts with explanations. Based on the Check-COVID dataset targeting misinformation during the pandemic.

radiospan