Skip to content
Showcase/GermanQuAD: German Extractive Question Answering
intermediatetext

GermanQuAD: German Extractive Question Answering

GermanQuAD is deepset's German machine reading comprehension dataset (MRQA 2021). Trained annotators read a German Wikipedia passage, write a self-sufficient question reformulated to reduce lexical overlap, and mark the answer span. 13,722 answerable QA pairs, with a runnable Potato config to reproduce or extend it.

About this dataset

GermanQuAD is a German extractive question answering dataset created by Möller, Risch, and Pietsch at deepset, published at the MRQA 2021 workshop. It provides a native (non-translated) German counterpart to SQuAD so that German QA and retrieval models can be trained and evaluated properly.

The passages come from German Wikipedia — the German equivalents of the articles used in English SQuAD — split into paragraphs. Rather than anonymous crowdworkers, the dataset used annotators familiar with machine learning who were given detailed guidelines, continuous monitoring, and expert review; one expert annotator alone produced more than 20,000 annotations.

The key protocol choice is fighting the lexical-overlap shortcut: annotators reformulate each question with synonyms and altered syntax, keep questions self-sufficient (answerable from the passage alone), and are encouraged to write complex questions whose answers span multiple sentences. The training set is one-way annotated; the test set carries three answers per question. GermanQuAD 1.0 has no unanswerable questions, and the release totals 13,722 QA pairs (train 11,518 / test 2,204).

The Potato config below reproduces this task: it shows a German Wikipedia passage, collects a self-sufficient written question, and captures the answer span. Use it to extend GermanQuAD, build a domain-specific German QA set, or run the low-overlap protocol in another language.

Released
MRQA 2021 (deepset)
Language
German
Source corpus
German Wikipedia
QA pairs
13,722 (train 11,518 / test 2,204)
Test answers
3-way annotated (6,536 answers)
Unanswerable questions
None (SQuAD 1.1 style)
PERORGLOCPERORGLOCDATESelect text to annotate

Configuration Fileconfig.yaml

This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.

yaml
# GermanQuAD - German Extractive Question Answering
# Based on Moeller et al., MRQA 2021
# Paper: https://aclanthology.org/2021.mrqa-1.4/
# Dataset: https://huggingface.co/datasets/deepset/germanquad
#
# GermanQuAD is the German extractive QA dataset from deepset. Trained
# annotators read a passage from German Wikipedia, write a self-sufficient
# question (deliberately reformulated to reduce lexical overlap with the
# passage), and mark the answer span. GermanQuAD 1.0 contains no
# SQuAD-2.0-style unanswerable questions.
#
# Annotation workflow (paper sections 2.1-2.3):
# 1. Read the German Wikipedia passage.
# 2. Write a self-sufficient question, reformulating with synonyms and altered
#    syntax so the answer cannot be found by simple word matching. Complex
#    questions whose answers span multiple sentences are encouraged.
# 3. Highlight the answer span in the passage.

annotation_task_name: "GermanQuAD - German Extractive QA"
task_dir: "."

data_files:
  - sample-data.json

item_properties:
  id_key: "id"
  text_key: "text"

output_annotation_dir: "annotation_output/"
output_annotation_format: "json"

port: 8000
server_name: localhost

annotation_schemes:
  # Step 1: Write a self-sufficient German question
  - annotation_type: text
    name: question
    description: "Formulieren Sie eine eigenständige Frage, die aus dem Absatz beantwortbar ist (Write a self-sufficient German question answerable from the passage)"

  # Step 2: Highlight the answer span
  - annotation_type: span
    name: answer_span
    description: "Markieren Sie die Textstelle, die die Frage beantwortet (Highlight the span that answers the question)"
    labels:
      - "Antwort (Answer)"
    label_colors:
      "Antwort (Answer)": "#3b82f6"

annotation_instructions: |
  You will be shown a passage from German Wikipedia. Following the GermanQuAD
  data-collection protocol (Möller et al., 2021):

  1. Read the passage carefully.
  2. Write a question in German that can be answered from the passage alone. Make
     it self-sufficient — a reader should not need any outside information. To
     avoid the lexical-overlap shortcut, reformulate the question using synonyms
     and altered syntax rather than reusing the passage's wording. Complex
     questions whose answers span multiple sentences are encouraged.
  3. Highlight the span in the passage that answers your question. Answers can
     range from a single entity to several sentences.

  Every question must be answerable from the passage — GermanQuAD 1.0 contains no
  unanswerable questions.

html_layout: |
  <div style="padding: 15px; max-width: 800px; margin: auto;">
    <div style="background: #f0f9ff; border: 1px solid #bae6fd; border-radius: 8px; padding: 16px; margin-bottom: 16px;">
      <strong style="color: #0369a1;">Absatz (Passage):</strong>
      <p style="font-size: 16px; line-height: 1.8; margin: 8px 0 0 0;">{{text}}</p>
    </div>
  </div>

allow_all_users: true
instances_per_annotator: 50
annotation_per_instance: 2
allow_skip: true
skip_reason_required: false

Sample Datasample-data.json

json
[
  {
    "id": "germanquad_001",
    "text": "Berlin ist die Hauptstadt und zugleich bevölkerungsreichste Stadt Deutschlands. Die Stadt liegt im Nordosten des Landes an den Flüssen Spree und Havel und hat rund 3,7 Millionen Einwohner. Berlin ist außerdem eines der sechzehn Bundesländer der Bundesrepublik Deutschland."
  },
  {
    "id": "germanquad_002",
    "text": "Johann Wolfgang von Goethe war ein deutscher Schriftsteller und Naturforscher. Sein bekanntestes Werk ist das Drama Faust, das er über mehrere Jahrzehnte hinweg verfasste. Goethe gilt als einer der bedeutendsten Vertreter der deutschsprachigen Literatur."
  }
]

// ... and 2 more items

Get This Design

View on GitHub

Clone or download from the repository

Quick start:

git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/text/reading-comprehension/germanquad-german-qa
potato start config.yaml

Dataset & paper

Möller et al., MRQA 2021

Citation (BibTeX)

bibtex
@inproceedings{moller-etal-2021-germanquad,
    title = "{G}erman{Q}u{AD} and {G}erman{DPR}: Improving Non-{E}nglish Question Answering and Passage Retrieval",
    author = "M{\"o}ller, Timo and Risch, Julian and Pietsch, Malte",
    booktitle = "Proceedings of the 3rd Workshop on Machine Reading for Question Answering",
    month = nov,
    year = "2021",
    address = "Punta Cana, Dominican Republic",
    publisher = "Association for Computational Linguistics",
    pages = "42--50",
    url = "https://aclanthology.org/2021.mrqa-1.4/"
}

Details

Annotation Types

textspan

Domain

Natural Language ProcessingQuestion AnsweringMultilingual

Use Cases

Reading ComprehensionExtractive QAGerman NLP

Tags

germanquadgermanquestion-answeringreading-comprehensionsquadextractive-qadeepset

Found an issue or want to improve this design?

Open an Issue