Skip to content
intermediateaudio

Speech Intelligibility Rating

Rate speech intelligibility for pathological speech following TORGO database annotation protocols.

Get This Design

This design is available in our showcase. Copy the configuration below to get started.

Quick start:

bash
# Create your project folder
mkdir speech-intelligibility-rating
cd speech-intelligibility-rating
# Save config.yaml from this page into this folder
potato start config.yaml
Speech Intelligibility Rating annotation interface preview

Configuration File

yaml
annotation_task_name: "Speech Intelligibility Rating"
task_dir: "."

port: 8000

# Data configuration
data_files:
  - "data/speech_samples.json"

item_properties:
  id_key: "id"
  text_key: "target_text"
instance_display:
  fields:
    - key: audio_url
      type: audio
      label: "Speech sample"
    - key: target_text
      type: text
      label: "Target phrase"

# Annotation schemes
annotation_schemes:
  # Transcription (orthographic)
  - annotation_type: text
    name: transcription
    description: "Write exactly what you hear (orthographic transcription)"
    textarea: false
    placeholder: "Type what you hear..."

  # Overall intelligibility
  - annotation_type: likert
    name: intelligibility
    description: "Overall speech intelligibility"
    size: 5
    labels:
      - "1: Unintelligible (cannot understand)"
      - "2: Mostly unintelligible"
      - "3: Partially intelligible"
      - "4: Mostly intelligible"
      - "5: Fully intelligible (clear)"

  # Severity rating (clinical scale)
  - annotation_type: radio
    name: severity
    description: "Speech disorder severity (if applicable)"
    labels:
      - Normal (no apparent disorder)
      - Mild (noticeable but easily understood)
      - Moderate (requires effort to understand)
      - Severe (very difficult to understand)
      - Profound (essentially unintelligible)

  # Articulation issues
  - annotation_type: multiselect
    name: articulation_issues
    description: "What articulation issues are present? (Select all)"
    labels:
      - Slurred consonants
      - Vowel distortions
      - Sound substitutions
      - Sound omissions
      - Hypernasality
      - Breathy voice
      - Strained voice
      - Monopitch (flat intonation)
      - Slow rate
      - Fast/rushed rate
      - Irregular rhythm
      - None apparent

  # Speech rate
  - annotation_type: radio
    name: speech_rate
    description: "How would you characterize the speech rate?"
    labels:
      - Much too slow
      - Somewhat slow
      - Normal rate
      - Somewhat fast
      - Much too fast/rushed

  # Effort to understand
  - annotation_type: likert
    name: listener_effort
    description: "How much effort was required to understand?"
    size: 5
    min_label: "No effort"
    max_label: "Extreme effort"

  # Speaker consistency
  - annotation_type: radio
    name: consistency
    description: "Was intelligibility consistent throughout?"
    labels:
      - Yes, consistent throughout
      - Variable (some parts clearer than others)
      - Progressively worse
      - Progressively better

  # Audio quality impact
  - annotation_type: radio
    name: quality_impact
    description: "Did recording quality affect your rating?"
    labels:
      - No (good quality recording)
      - Minor impact
      - Significant impact
      - Cannot rate due to poor quality

  # Confidence
  - annotation_type: likert
    name: confidence
    description: "Confidence in your intelligibility rating"
    size: 5
    min_label: "Low"
    max_label: "High"

# User settings
require_password: false

# Output
output_annotation_dir: "annotation_output/"
export_annotation_format: "jsonl"

Sample Data

json
[
  {
    "id": "speech_intelligibility_rating_1",
    "audio_url": "https://upload.wikimedia.org/wikipedia/commons/0/02/En-us-See_the_saw_you_saw.ogg",
    "target_text": "See the saw you saw."
  },
  {
    "id": "speech_intelligibility_rating_2",
    "audio_url": "https://upload.wikimedia.org/wikipedia/commons/f/fa/En-us-the_buck_stops_here.oga",
    "target_text": "The buck stops here."
  }
]

// ... and 3 more items

Details

Annotation Types

likertradiotext

Domain

AudioSpeechMedical

Use Cases

speech pathologyintelligibility ratingdysarthria assessment

Tags

audiospeech pathologyintelligibilitydysarthriaclinical