AnuraSet Frog Call Segments and Calling Intensity

# yaml-language-server: $schema=https://potatoannotator.readthedocs.io/en/latest/schemas/potato-config.schema.json
# AnuraSet frog calls: call segments and per-species calling intensity.
#
# Dataset: AnuraSet, a benchmark for Neotropical anuran call identification in
#   passive acoustic monitoring (1,612 one-minute recordings, 42 species, four
#   monitoring sites in two Brazilian biomes; Cañas, Toro-Gómez, Sugai, and
#   colleagues).
# Paper:   Scientific Data 10, 771 (2023). DOI: 10.1038/s41597-023-02666-2
#   @article{Canas_2023,
#     title   = {A dataset for benchmarking Neotropical anuran calls
#                identification in passive acoustic monitoring},
#     volume  = {10}, number = {1}, journal = {Scientific Data},
#     publisher = {Springer Science and Business Media LLC},
#     author  = {Cañas, Juan Sebastián and Toro-Gómez, María Paula and Sugai,
#                Larissa Sayuri Moreira and Benítez Restrepo, Hernán Darío and
#                Rudas, Jorge and Posso Bautista, Breyner and Toledo, Luís Felipe
#                and Dena, Simone and Domingos, Adão Henrique Rosa and de Souza,
#                Franco Leandro and Neckel-Oliveira, Selvino and da Rosa, Anderson
#                and Carvalho-Rocha, Vítor and Bernardy, José Vinícius and Sugai,
#                José Luiz Massao Moreira and dos Santos, Carolina Emília and
#                Bastos, Rogério Pereira and Llusia, Diego and Ulloa, Juan
#                Sebastián},
#     year    = {2023}, month = nov, DOI = {10.1038/s41597-023-02666-2}}
# Data:    https://zenodo.org/records/8342596 (DOI 10.5281/zenodo.8342596), the
#   record the article's Code availability section cites. An older record,
#   https://zenodo.org/records/8043210 (DOI 10.5281/zenodo.8043210), holds an
#   earlier release of the same files.
#
# Licence: the data is treated here as CC BY. The article's Data Records section
#   says "The dataset and the raw data are provided under the Public Domain
#   Dedication license (CC0)", while the Zenodo records (8043210 and the latest
#   8342596) list Creative Commons Attribution, and DataCite gives "Creative
#   Commons Attribution 1.0 Generic". The two disagree, so the stricter terms
#   apply and the media in media/ carries attribution (see LICENSE-media.txt).
#   The ten WAV files were cut from raw_data.zip and re-encoded as 48 kbps mono
#   MP3 (about 360 KB each); the recordings are real.
#
# Protocol, quoted from the article (Methods, "Audio Annotation"):
#   Step 2, weak labeling: "Local annotators were asked to report the level of
#   calling activity of each recorded anuran species based on the Amphibian
#   Calling Index (Table 1), according to the species-specific calling activity
#   level in each 1-minute audio file (weak labeling)."
#   Table 1, levels of anuran calling activity for the weak labeling:
#     Absence (0): No anuran calls recorded
#     Low (1): Individual anurans can be counted and calls do not overlap
#     Moderate (2): Few anuran individuals that cannot be accurately counted, and
#       with both separate and overlapping calls
#     High (3): Intense chorus, with continuous and overlapping calls from
#       different individuals
#   Step 3, strong labeling: "Using Audacity 3.2, we conducted a detailed visual
#   and aural inspection of the spectrogram to identify temporal limits
#   (beginning and end) containing species-specific calls with an inter-call
#   interval of less than 1 second." "For longer inter-call intervals, we boxed
#   calls separately and labeled them independently." Each time box carries "the
#   species ID, tagged with a unique 6-letter code built from the scientific name
#   of each identified species".
#
# Label glossary (code, species, from Supplementary Table 1 of the article):
#   DENMIN Dendropsophus minutus
#   BOABIS Boana bischoffi
#   PHYALB Physalaemus albonotatus
#   SPHSUR Sphaenorhynchus surdus
#   LEPLAT Leptodactylus latrans
#   PITAZU Pithecopus azureus
#   BOAALB Boana albopunctata
#   LEPPOD Leptodactylus podicipinus
#
# Adaptations:
#   - The original has 42 species. This design keeps the eight with the most
#     time boxes in the dataset's strong_labels files (DENMIN, BOABIS, PHYALB,
#     SPHSUR, LEPLAT, PITAZU, BOAALB, and LEPPOD, in that order; per-species
#     box counts are not given here because two counts of the same files
#     differ by a few boxes). The ten sample recordings were chosen
#     so that every strong label in them is one of these eight, so no box is
#     dropped.
#   - The original weak labels were made in Audacity by local herpetologists and
#     the strong labels by one herpetologist in a separate pass. Here both sit
#     on one page: segments on the waveform and one 0-3 radio per species.
#   - The dataset's strong labels are loaded as pre-annotations (the "calls"
#     scheme), so annotators correct them. The radios start empty. The released
#     weak_labels.csv is not shown.
#   - The released strong-label files append a signal-quality letter to each
#     species code (for example PITAZU_M). The article describes the quality
#     letter as L, M, or H. The later record 8342596 uses L, M, and H, as the
#     article does. The older record 8043210 uses F, M, and C instead (and 21
#     labels that read FALSE). The quality tag is left out of this design.
#   - Only temporal limits were marked in the original, so the segments carry no
#     frequency bounds, and the waveform display is used instead of a box.
#   - The sample is ten full one-minute recordings, one of them (site INCT04 in the article,
#     INCT4 in the file names; 2019-10-23 00:15) with no calls in the dataset's labels.
#
# Run from this folder:
#   potato start config.yaml -p 8000
#
# Training (optional). The hf-audio-tagger trainer fine-tunes a wav2vec 2.0 style
# speech encoder (default facebook/wav2vec2-base) to say which species label is
# active in each frame of about 20 ms, from the segments annotators have
# confirmed. Predictions are written only onto recordings outside the training
# split, and there they replace the dataset segments loaded into the same
# predictions field. It trains on the "calls" segments only; the radios are not
# used. It does not serve or deploy the model. It needs
# `pip install 'potato-annotation[train-audio]'`. Open /admin/training with the key
# below, pick hf-audio-tagger and the calls scheme, and click Train now. The
# trainer asks for at least 10 recordings in its train split by default, so ten
# sample recordings are a demonstration, not enough to train a useful tagger.
port: 8000
annotation_task_name: "AnuraSet frog calls: segments and calling intensity"
task_dir: "."
output_annotation_dir: "annotation_output"
export_annotation_format: jsonl
data_files:
  - "sample-data.json"
item_properties:
  id_key: id
  text_key: audio_url

require_password: false
admin_api_key: "demo-admin-key"

user_config:
  allow_all_users: true
  users: []

instance_display:
  fields:
    - key: recording
      type: text
      label: Recording

pre_annotation:
  enabled: true
  field: predictions
  allow_modification: true

audio_annotation:
  waveform_cache_dir: waveform_cache/
  waveform_look_ahead: 3
  waveform_cache_max_size: 50

annotation_schemes:
  - annotation_type: audio_annotation
    name: calls
    description: "Check the call segments loaded from the dataset's strong labels. Each segment marks the beginning and end of a species-specific bout of advertisement calls. Move, resize, relabel or delete them, and mark any bout that was missed."
    mode: label
    labels:
      - name: DENMIN
        color: "#2563eb"
        key_value: "1"
      - name: BOABIS
        color: "#dc2626"
        key_value: "2"
      - name: PHYALB
        color: "#16a34a"
        key_value: "3"
      - name: SPHSUR
        color: "#d97706"
        key_value: "4"
      - name: LEPLAT
        color: "#9333ea"
        key_value: "5"
      - name: PITAZU
        color: "#0891b2"
        key_value: "6"
      - name: BOAALB
        color: "#db2777"
        key_value: "7"
      - name: LEPPOD
        color: "#65a30d"
        key_value: "8"
    min_segments: 0
    zoom_enabled: true
    playback_rate_control: true
    spectrogram: true

  - annotation_type: radio
    name: intensity_DENMIN
    description: "DENMIN (Dendropsophus minutus): calling activity level in this 1-minute recording. Absence (0): No anuran calls recorded. Low (1): Individual anurans can be counted and calls do not overlap. Moderate (2): Few anuran individuals that cannot be accurately counted, and with both separate and overlapping calls. High (3): Intense chorus, with continuous and overlapping calls from different individuals."
    horizontal: true
    labels:
      - name: "0 Absence"
        tooltip: "Absence (0): No anuran calls recorded"
      - name: "1 Low"
        tooltip: "Low (1): Individual anurans can be counted and calls do not overlap"
      - name: "2 Moderate"
        tooltip: "Moderate (2): Few anuran individuals that cannot be accurately counted, and with both separate and overlapping calls"
      - name: "3 High"
        tooltip: "High (3): Intense chorus, with continuous and overlapping calls from different individuals"

  - annotation_type: radio
    name: intensity_BOABIS
    description: "BOABIS (Boana bischoffi): calling activity level in this 1-minute recording. Absence (0): No anuran calls recorded. Low (1): Individual anurans can be counted and calls do not overlap. Moderate (2): Few anuran individuals that cannot be accurately counted, and with both separate and overlapping calls. High (3): Intense chorus, with continuous and overlapping calls from different individuals."
    horizontal: true
    labels:
      - name: "0 Absence"
        tooltip: "Absence (0): No anuran calls recorded"
      - name: "1 Low"
        tooltip: "Low (1): Individual anurans can be counted and calls do not overlap"
      - name: "2 Moderate"
        tooltip: "Moderate (2): Few anuran individuals that cannot be accurately counted, and with both separate and overlapping calls"
      - name: "3 High"
        tooltip: "High (3): Intense chorus, with continuous and overlapping calls from different individuals"

  - annotation_type: radio
    name: intensity_PHYALB
    description: "PHYALB (Physalaemus albonotatus): calling activity level in this 1-minute recording. Absence (0): No anuran calls recorded. Low (1): Individual anurans can be counted and calls do not overlap. Moderate (2): Few anuran individuals that cannot be accurately counted, and with both separate and overlapping calls. High (3): Intense chorus, with continuous and overlapping calls from different individuals."
    horizontal: true
    labels:
      - name: "0 Absence"
        tooltip: "Absence (0): No anuran calls recorded"
      - name: "1 Low"
        tooltip: "Low (1): Individual anurans can be counted and calls do not overlap"
      - name: "2 Moderate"
        tooltip: "Moderate (2): Few anuran individuals that cannot be accurately counted, and with both separate and overlapping calls"
      - name: "3 High"
        tooltip: "High (3): Intense chorus, with continuous and overlapping calls from different individuals"

  - annotation_type: radio
    name: intensity_SPHSUR
    description: "SPHSUR (Sphaenorhynchus surdus): calling activity level in this 1-minute recording. Absence (0): No anuran calls recorded. Low (1): Individual anurans can be counted and calls do not overlap. Moderate (2): Few anuran individuals that cannot be accurately counted, and with both separate and overlapping calls. High (3): Intense chorus, with continuous and overlapping calls from different individuals."
    horizontal: true
    labels:
      - name: "0 Absence"
        tooltip: "Absence (0): No anuran calls recorded"
      - name: "1 Low"
        tooltip: "Low (1): Individual anurans can be counted and calls do not overlap"
      - name: "2 Moderate"
        tooltip: "Moderate (2): Few anuran individuals that cannot be accurately counted, and with both separate and overlapping calls"
      - name: "3 High"
        tooltip: "High (3): Intense chorus, with continuous and overlapping calls from different individuals"

  - annotation_type: radio
    name: intensity_LEPLAT
    description: "LEPLAT (Leptodactylus latrans): calling activity level in this 1-minute recording. Absence (0): No anuran calls recorded. Low (1): Individual anurans can be counted and calls do not overlap. Moderate (2): Few anuran individuals that cannot be accurately counted, and with both separate and overlapping calls. High (3): Intense chorus, with continuous and overlapping calls from different individuals."
    horizontal: true
    labels:
      - name: "0 Absence"
        tooltip: "Absence (0): No anuran calls recorded"
      - name: "1 Low"
        tooltip: "Low (1): Individual anurans can be counted and calls do not overlap"
      - name: "2 Moderate"
        tooltip: "Moderate (2): Few anuran individuals that cannot be accurately counted, and with both separate and overlapping calls"
      - name: "3 High"
        tooltip: "High (3): Intense chorus, with continuous and overlapping calls from different individuals"

  - annotation_type: radio
    name: intensity_PITAZU
    description: "PITAZU (Pithecopus azureus): calling activity level in this 1-minute recording. Absence (0): No anuran calls recorded. Low (1): Individual anurans can be counted and calls do not overlap. Moderate (2): Few anuran individuals that cannot be accurately counted, and with both separate and overlapping calls. High (3): Intense chorus, with continuous and overlapping calls from different individuals."
    horizontal: true
    labels:
      - name: "0 Absence"
        tooltip: "Absence (0): No anuran calls recorded"
      - name: "1 Low"
        tooltip: "Low (1): Individual anurans can be counted and calls do not overlap"
      - name: "2 Moderate"
        tooltip: "Moderate (2): Few anuran individuals that cannot be accurately counted, and with both separate and overlapping calls"
      - name: "3 High"
        tooltip: "High (3): Intense chorus, with continuous and overlapping calls from different individuals"

  - annotation_type: radio
    name: intensity_BOAALB
    description: "BOAALB (Boana albopunctata): calling activity level in this 1-minute recording. Absence (0): No anuran calls recorded. Low (1): Individual anurans can be counted and calls do not overlap. Moderate (2): Few anuran individuals that cannot be accurately counted, and with both separate and overlapping calls. High (3): Intense chorus, with continuous and overlapping calls from different individuals."
    horizontal: true
    labels:
      - name: "0 Absence"
        tooltip: "Absence (0): No anuran calls recorded"
      - name: "1 Low"
        tooltip: "Low (1): Individual anurans can be counted and calls do not overlap"
      - name: "2 Moderate"
        tooltip: "Moderate (2): Few anuran individuals that cannot be accurately counted, and with both separate and overlapping calls"
      - name: "3 High"
        tooltip: "High (3): Intense chorus, with continuous and overlapping calls from different individuals"

  - annotation_type: radio
    name: intensity_LEPPOD
    description: "LEPPOD (Leptodactylus podicipinus): calling activity level in this 1-minute recording. Absence (0): No anuran calls recorded. Low (1): Individual anurans can be counted and calls do not overlap. Moderate (2): Few anuran individuals that cannot be accurately counted, and with both separate and overlapping calls. High (3): Intense chorus, with continuous and overlapping calls from different individuals."
    horizontal: true
    labels:
      - name: "0 Absence"
        tooltip: "Absence (0): No anuran calls recorded"
      - name: "1 Low"
        tooltip: "Low (1): Individual anurans can be counted and calls do not overlap"
      - name: "2 Moderate"
        tooltip: "Moderate (2): Few anuran individuals that cannot be accurately counted, and with both separate and overlapping calls"
      - name: "3 High"
        tooltip: "High (3): Intense chorus, with continuous and overlapping calls from different individuals"

annotation_instructions: "Listen to each one-minute recording from a Brazilian frog survey. Check the call segments, which mark where each species is calling, and fix wrong labels, edges or missing bouts for the eight listed species. Then rate each species' calling activity for the whole minute from 0 (absence) to 3 (intense chorus)."

model_training:
  enabled: true
  writeback:
    schemas: ["calls"]
    include_unlabeled: true
    min_confidence: 0.5