ODAQ MUSHRA Listening Test for Audio Quality

# ODAQ: a MUSHRA listening test on the Open Dataset of Audio Quality
#
# Dataset: ODAQ (Open Dataset of Audio Quality), Fraunhofer IIS and Netflix.
# Paper: Torcoli, M., Wu, C.-W., Dick, S., Williams, P. A., Halimeh, M. M.,
#   Wolcott, W., and Habets, E. A. P. (2024). ODAQ: Open Dataset of Audio
#   Quality. ICASSP 2024, pp. 836-840. Preprint: https://arxiv.org/abs/2401.00197
# BibTeX (Crossref):
#   @inproceedings{Torcoli_2024, title={Odaq: Open Dataset of Audio Quality},
#     DOI={10.1109/icassp48485.2024.10447634},
#     booktitle={ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
#     publisher={IEEE},
#     author={Torcoli, Matteo and Wu, Chih-Wei and Dick, Sascha and Williams, Phillip A. and Modar Halimeh, Mhd and Wolcott, William and Habets, Emanuël A. P.},
#     year={2024}, month=Apr, pages={836-840}}
# DOI: 10.1109/ICASSP48485.2024.10447634
# Dataset: https://zenodo.org/records/10405774 (DOI 10.5281/zenodo.10405774)
# Licence: ODAQ mixes CC BY, CC BY-NC and CC0 audio, per original file
#   (_detailed_license.csv). The five trials in media/ use only CC BY 4.0 and
#   CC0 originals, each credited in media/LICENSE-media.txt.
#
# Protocol (Torcoli et al. 2024, Sections 3.1 and 4.1, arXiv v1):
#   "ODAQ consists of 30 trials; each trial consists of one source (i.e.,
#   reference), two low-pass anchors as recommended by [4], and 5 processed
#   samples." ([4] is ITU-R BS.1534.)
#   "The 6240 subjective scores are Basic Audio Quality (BAQ) scores collected
#   via an extensive MUSHRA test [4] involving 26 expert listeners."
#   "15 listeners participated in Germany, while 13 listeners participated in
#   the US." One listener per laboratory was excluded "for having rated the
#   hidden reference below 90 MUSHRA points for more than 15% of all trials".
#   Before the 30 test trials (3 sessions of 10), listeners did a training
#   session of "3 trials with audio samples that are not used in the full test".
#   The anchors are low-pass filtered references at 3.5 and 7 kHz (the paper's
#   LP3.5 and LP7). The score axis in the paper's results figure runs 0 to 100
#   with the five BS.1534 quality labels Excellent, Good, Fair, Poor, and Bad.
#
# Scale: Potato's mushra method shows one 0-100 slider per sample, labeled
#   Excellent, Good, Fair, Poor, and Bad, and it adds the hidden
#   reference and the two low-pass anchors itself. Samples are shuffled per
#   listener, and playback switches between them at the same position, with an
#   optional loop. By default the listener must hear every sample, and one
#   sample must be rated 100 (require_one_at_100) before the page accepts the
#   ratings.
#
# Adaptations:
#   - Five trials, not 30, each cut to a 2 s window of the trial files and
#     re-encoded (16-bit; WAV reference, FLAC processed files) to keep media/
#     near 5 MB. The five trial files used here are 7.5 to 11.5 s long (ODAQ
#     averages 12.5 s). Only CC BY and CC0 originals are used.
#     The trials cover five of the six method classes (TM, SH, PE, LP, UN).
#     The dialogue enhancement class (DE) is left out to keep the demo small.
#   - Each trial carries the five processed files from ODAQ's trial folder.
#     ODAQ's two anchor files are not shipped. Potato makes its own 3.5 and
#     7 kHz low-pass anchors from the reference (brick-wall filter with a
#     200 Hz taper), so the anchors are similar to, not identical with, ODAQ's.
#   - ODAQ ran in a purpose-built JUCE application in acoustically damped
#     listening rooms on closed-back headphones. This design runs in the
#     browser, and the listener's equipment is not controlled.
#   - The 3-trial training session and the split into three sessions are not
#     reproduced (a training page would be a `phases` entry). The paper's
#     post-screening rule is not applied automatically here. The export below
#     applies Potato's BS.1534-3 post-screening to the collected ratings.
#   - The paper does not print its listener instructions, but the Zenodo
#     package ships them (listening_test_instructions/
#     listening_test_instructions_english.docx, ODAQ's authors). They tell
#     listeners to rate "overall sound quality considering any and all detected
#     differences with the reference" on "a continuous scale varying between 0
#     and 100", and to rate the hidden reference "with 100 points". The wording
#     of `instructions` and `annotation_instructions` below follows that
#     sense but is written for this design and is not ODAQ's text.
#   - Condition names in the data file (tm3k, sh10_ms, and so on) are ODAQ's
#     file names. Listeners never see them, since Potato presents opaque slots.
#   - No model_training block. No registered Potato trainer consumes a
#     listening_test scheme, so the ratings are exported for analysis only:
#       python -m potato.export --config config.yaml --format listening_test --output out/
#     writes ratings by condition, with means, 95% intervals and the
#     post-screening.
#
# Run: potato start config.yaml -p 8000

annotation_task_name: "ODAQ MUSHRA listening test"

data_files:
  - sample-data.json

item_properties:
  id_key: id
  text_key: text

task_dir: .
output_annotation_dir: annotation_output
export_annotation_format: jsonl
media_directory: media

user_config:
  allow_all_users: true
  users: []

annotation_instructions: >-
  Each trial has an open reference and several samples. Listen to the
  reference, then to every sample, and score the Basic Audio Quality of each
  sample against the reference on the 0 to 100 scale. One sample is the
  reference itself and should be scored 100.

annotation_schemes:
  - annotation_type: listening_test
    name: baq
    method: mushra
    description: "Rate the basic audio quality of each sample against the reference"
    instructions: >-
      Switch between the reference and the samples as often as you like, and
      use the loop to compare a short passage. Score the Basic Audio Quality of
      every sample from 0 (Bad) to 100 (Excellent). One of the samples is the
      hidden reference.
    reference_field: reference
    stimuli_field: stimuli
    anchors: [3500, 7000]
    hidden_reference: true
    loop: true
    label_requirement:
      required: true