GermanQuAD: German Extractive Question Answering
GermanQuAD is deepset's German machine reading comprehension dataset (MRQA 2021). Trained annotators read a German Wikipedia passage, write a self-sufficient question reformulated to reduce lexical overlap, and mark the answer span. 13,722 answerable QA pairs, with a runnable Potato config to reproduce or extend it.
About this dataset
GermanQuAD is a German extractive question answering dataset created by Möller, Risch, and Pietsch at deepset, published at the MRQA 2021 workshop. It provides a native (non-translated) German counterpart to SQuAD so that German QA and retrieval models can be trained and evaluated properly.
The passages come from German Wikipedia — the German equivalents of the articles used in English SQuAD — split into paragraphs. Rather than anonymous crowdworkers, the dataset used annotators familiar with machine learning who were given detailed guidelines, continuous monitoring, and expert review; one expert annotator alone produced more than 20,000 annotations.
The key protocol choice is fighting the lexical-overlap shortcut: annotators reformulate each question with synonyms and altered syntax, keep questions self-sufficient (answerable from the passage alone), and are encouraged to write complex questions whose answers span multiple sentences. The training set is one-way annotated; the test set carries three answers per question. GermanQuAD 1.0 has no unanswerable questions, and the release totals 13,722 QA pairs (train 11,518 / test 2,204).
The Potato config below reproduces this task: it shows a German Wikipedia passage, collects a self-sufficient written question, and captures the answer span. Use it to extend GermanQuAD, build a domain-specific German QA set, or run the low-overlap protocol in another language.
- Released
- MRQA 2021 (deepset)
- Language
- German
- Source corpus
- German Wikipedia
- QA pairs
- 13,722 (train 11,518 / test 2,204)
- Test answers
- 3-way annotated (6,536 answers)
- Unanswerable questions
- None (SQuAD 1.1 style)
Configuration Fileconfig.yaml
This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.
# GermanQuAD - German Extractive Question Answering
# Based on Moeller et al., MRQA 2021
# Paper: https://aclanthology.org/2021.mrqa-1.4/
# Dataset: https://huggingface.co/datasets/deepset/germanquad
#
# GermanQuAD is the German extractive QA dataset from deepset. Trained
# annotators read a passage from German Wikipedia, write a self-sufficient
# question (deliberately reformulated to reduce lexical overlap with the
# passage), and mark the answer span. GermanQuAD 1.0 contains no
# SQuAD-2.0-style unanswerable questions.
#
# Annotation workflow (paper sections 2.1-2.3):
# 1. Read the German Wikipedia passage.
# 2. Write a self-sufficient question, reformulating with synonyms and altered
# syntax so the answer cannot be found by simple word matching. Complex
# questions whose answers span multiple sentences are encouraged.
# 3. Highlight the answer span in the passage.
annotation_task_name: "GermanQuAD - German Extractive QA"
task_dir: "."
data_files:
- sample-data.json
item_properties:
id_key: "id"
text_key: "text"
output_annotation_dir: "annotation_output/"
output_annotation_format: "json"
port: 8000
server_name: localhost
annotation_schemes:
# Step 1: Write a self-sufficient German question
- annotation_type: text
name: question
description: "Formulieren Sie eine eigenständige Frage, die aus dem Absatz beantwortbar ist (Write a self-sufficient German question answerable from the passage)"
# Step 2: Highlight the answer span
- annotation_type: span
name: answer_span
description: "Markieren Sie die Textstelle, die die Frage beantwortet (Highlight the span that answers the question)"
labels:
- "Antwort (Answer)"
label_colors:
"Antwort (Answer)": "#3b82f6"
annotation_instructions: |
You will be shown a passage from German Wikipedia. Following the GermanQuAD
data-collection protocol (Möller et al., 2021):
1. Read the passage carefully.
2. Write a question in German that can be answered from the passage alone. Make
it self-sufficient — a reader should not need any outside information. To
avoid the lexical-overlap shortcut, reformulate the question using synonyms
and altered syntax rather than reusing the passage's wording. Complex
questions whose answers span multiple sentences are encouraged.
3. Highlight the span in the passage that answers your question. Answers can
range from a single entity to several sentences.
Every question must be answerable from the passage — GermanQuAD 1.0 contains no
unanswerable questions.
html_layout: |
<div style="padding: 15px; max-width: 800px; margin: auto;">
<div style="background: #f0f9ff; border: 1px solid #bae6fd; border-radius: 8px; padding: 16px; margin-bottom: 16px;">
<strong style="color: #0369a1;">Absatz (Passage):</strong>
<p style="font-size: 16px; line-height: 1.8; margin: 8px 0 0 0;">{{text}}</p>
</div>
</div>
allow_all_users: true
instances_per_annotator: 50
annotation_per_instance: 2
allow_skip: true
skip_reason_required: false
Sample Datasample-data.json
[
{
"id": "germanquad_001",
"text": "Berlin ist die Hauptstadt und zugleich bevölkerungsreichste Stadt Deutschlands. Die Stadt liegt im Nordosten des Landes an den Flüssen Spree und Havel und hat rund 3,7 Millionen Einwohner. Berlin ist außerdem eines der sechzehn Bundesländer der Bundesrepublik Deutschland."
},
{
"id": "germanquad_002",
"text": "Johann Wolfgang von Goethe war ein deutscher Schriftsteller und Naturforscher. Sein bekanntestes Werk ist das Drama Faust, das er über mehrere Jahrzehnte hinweg verfasste. Goethe gilt als einer der bedeutendsten Vertreter der deutschsprachigen Literatur."
}
]
// ... and 2 more itemsGet This Design
Clone or download from the repository
Quick start:
git clone https://github.com/davidjurgens/potato-showcase.git cd potato-showcase/text/reading-comprehension/germanquad-german-qa potato start config.yaml
Dataset & paper
Möller et al., MRQA 2021
Citation (BibTeX)
@inproceedings{moller-etal-2021-germanquad,
title = "{G}erman{Q}u{AD} and {G}erman{DPR}: Improving Non-{E}nglish Question Answering and Passage Retrieval",
author = "M{\"o}ller, Timo and Risch, Julian and Pietsch, Malte",
booktitle = "Proceedings of the 3rd Workshop on Machine Reading for Question Answering",
month = nov,
year = "2021",
address = "Punta Cana, Dominican Republic",
publisher = "Association for Computational Linguistics",
pages = "42--50",
url = "https://aclanthology.org/2021.mrqa-1.4/"
}Details
Annotation Types
Domain
Use Cases
Tags
Found an issue or want to improve this design?
Open an IssueRelated Designs
FQuAD: French Question Answering Dataset
FQuAD is the French extractive QA dataset from Illuin Technology, built on the SQuAD 1.1 protocol. University students read a French Wikipedia paragraph, write a question, and select the smallest answer span. Over 60,000 answerable QA pairs, with a runnable Potato config to reproduce or extend it.
KorQuAD: Korean Extractive Question Answering
KorQuAD 1.0 is the Korean machine reading comprehension dataset built on the SQuAD 1.0 protocol: crowdworkers read a Korean Wikipedia passage, write a question, and mark the minimal answer span. 70,079 answerable QA pairs. Includes a runnable Potato config to reproduce or extend the annotation.
Natural Questions - Open-Domain Question Answering
Open-domain question answering over Wikipedia passages, based on Google's Natural Questions dataset (Kwiatkowski et al., TACL 2019). Annotators identify both short and long answer spans and determine answerability.