語音情感識別（SER）為虛擬助手、心理健康應用和客戶服務分析提供支援。本教程展示如何構建分類情感、維度評分和混合情感檢測的標註介面。

情感標註方法

標註語音情感有幾種方式：

分類式：離散標籤（快樂、悲傷、憤怒）
維度式：連續量表（效價、喚醒度、支配度）
混合式：多種情感及強度評分
片段式：不同時間戳對應不同情感

分類式情感分類

基本配置

yaml

annotation_task_name: "Speech Emotion Recognition"
 
data_files:
  - data/utterances.json
 
item_properties:
  id_key: id
  audio_key: audio_path
  text_key: transcript  # Optional transcript
 
audio:
  enabled: true
  display: waveform
  waveform_color: "#8B5CF6"
  progress_color: "#A78BFA"
  speed_control: true
  speed_options: [0.75, 1.0, 1.25]
 
annotation_schemes:
  - annotation_type: radio
    name: emotion
    description: "What emotion is expressed in this speech?"
    labels:
      - name: Happy
        description: "Joy, excitement, amusement"
        keyboard_shortcut: "h"
      - name: Sad
        description: "Sorrow, disappointment, grief"
        keyboard_shortcut: "s"
      - name: Angry
        description: "Frustration, irritation, rage"
        keyboard_shortcut: "a"
      - name: Fearful
        description: "Anxiety, worry, terror"
        keyboard_shortcut: "f"
      - name: Surprised
        description: "Astonishment, shock"
        keyboard_shortcut: "u"
      - name: Disgusted
        description: "Revulsion, distaste"
        keyboard_shortcut: "d"
      - name: Neutral
        description: "No clear emotion"
        keyboard_shortcut: "n"
    label_requirement:
      required: true

新增強度評分

yaml

annotation_schemes:
  - annotation_type: radio
    name: emotion
    labels: [Happy, Sad, Angry, Fearful, Surprised, Disgusted, Neutral]
    label_requirement:
      required: true
 
  - annotation_type: likert
    name: intensity
    description: "How intense is this emotion?"
    size: 5
    min_label: "Very weak"
    max_label: "Very strong"

維度式情感標註

VAD（效價-喚醒度-支配度）模型：

yaml

annotation_task_name: "Dimensional Emotion Rating"
 
annotation_schemes:
  # Valence: negative to positive
  - annotation_type: likert
    name: valence
    description: "Valence: How positive or negative?"
    size: 7
    min_label: "Very negative"
    max_label: "Very positive"
 
  # Arousal: calm to excited
  - annotation_type: likert
    name: arousal
    description: "Arousal: How calm or excited?"
    size: 7
    min_label: "Very calm"
    max_label: "Very excited"
 
  # Dominance: submissive to dominant
  - annotation_type: likert
    name: dominance
    description: "Dominance: How submissive or dominant?"
    size: 7
    min_label: "Very submissive"
    max_label: "Very dominant"

視覺量表 (SAM)

自我評估人偶（Self-Assessment Manikin）風格：

yaml

annotation_schemes:
  - annotation_type: likert
    min_label: "Very negative"
    max_label: "Very positive"
    name: valence
    description: "Select the figure that matches the emotional valence"
    size: 9

混合情感檢測

對於包含多種情感的語音：

yaml

annotation_schemes:
  - annotation_type: multiselect
    name: emotions_present
    description: "Select ALL emotions you detect (can be multiple)"
    labels:
      - Happy
      - Sad
      - Angry
      - Fearful
      - Surprised
      - Disgusted
      - Contempt
 
  - annotation_type: radio
    name: primary_emotion
    description: "Which emotion is MOST prominent?"
    labels:
      - Happy
      - Sad
      - Angry
      - Fearful
      - Surprised
      - Disgusted
      - Contempt
      - Mixed (no dominant)

綜合情感標註

yaml

annotation_task_name: "Comprehensive Speech Emotion Annotation"
 
data_files:
  - data/speech_samples.json
 
item_properties:
  id_key: id
  audio_key: audio_url
  text_key: transcript
 
audio:
  enabled: true
  display: waveform
  waveform_color: "#EC4899"
  progress_color: "#F472B6"
  height: 120
  speed_control: true
  speed_options: [0.5, 0.75, 1.0, 1.25]
  show_duration: true
  autoplay: false
 
# Show transcript if available
display:
  show_text: true
  text_field: transcript
  text_label: "Transcript (for reference)"
 
annotation_schemes:
  # Primary categorical emotion
  - annotation_type: radio
    name: primary_emotion
    description: "Primary emotion expressed"
    labels:
      - name: Happiness
        color: "#FCD34D"
        keyboard_shortcut: "1"
      - name: Sadness
        color: "#60A5FA"
        keyboard_shortcut: "2"
      - name: Anger
        color: "#F87171"
        keyboard_shortcut: "3"
      - name: Fear
        color: "#A78BFA"
        keyboard_shortcut: "4"
      - name: Surprise
        color: "#34D399"
        keyboard_shortcut: "5"
      - name: Disgust
        color: "#FB923C"
        keyboard_shortcut: "6"
      - name: Neutral
        color: "#9CA3AF"
        keyboard_shortcut: "7"
    label_requirement:
      required: true
 
  # Emotional intensity
  - annotation_type: likert
    name: intensity
    description: "Emotional intensity"
    size: 5
    min_label: "Very mild"
    max_label: "Very intense"
    label_requirement:
      required: true
 
  # Dimensional ratings
  - annotation_type: likert
    name: valence
    description: "Valence (negative to positive)"
    size: 7
    min_label: "Negative"
    max_label: "Positive"
 
  - annotation_type: likert
    name: arousal
    description: "Arousal (calm to excited)"
    size: 7
    min_label: "Calm"
    max_label: "Excited"
 
  # Voice quality
  - annotation_type: multiselect
    name: voice_qualities
    description: "Voice characteristics (select all that apply)"
    labels:
      - Trembling voice
      - Raised pitch
      - Lowered pitch
      - Loud/shouting
      - Soft/whisper
      - Fast speech rate
      - Slow speech rate
      - Breathy
      - Tense/strained
      - Crying
      - Laughing
 
  # Genuineness
  - annotation_type: radio
    name: authenticity
    description: "Does the emotion seem genuine?"
    labels:
      - Clearly genuine
      - Likely genuine
      - Uncertain
      - Likely acted/fake
      - Clearly acted/fake
 
  # Confidence
  - annotation_type: likert
    name: confidence
    description: "How confident are you in your annotation?"
    size: 5
    min_label: "Guessing"
    max_label: "Certain"
 
annotation_guidelines:
  title: "Emotion Annotation Guidelines"
  content: |
    ## Listening Instructions
    1. Listen to the entire clip before annotating
    2. You may replay as many times as needed
    3. Focus on the VOICE, not just the words
 
    ## Emotion Categories
    - **Happiness**: Joy, amusement, contentment
    - **Sadness**: Sorrow, disappointment, melancholy
    - **Anger**: Frustration, irritation, rage
    - **Fear**: Anxiety, nervousness, terror
    - **Surprise**: Astonishment, startle
    - **Disgust**: Revulsion, contempt
    - **Neutral**: Calm, matter-of-fact
 
    ## Tips
    - Consider tone, pitch, speaking rate
    - The transcript may not match the emotion
    - When unsure between two emotions, choose the stronger one
    - Use the intensity scale for unclear cases
 
output_annotation_dir: annotations/
export_annotation_format: jsonl

輸出格式

json

{
  "id": "utt_001",
  "audio_url": "/audio/sample_001.wav",
  "transcript": "I can't believe this happened!",
  "annotations": {
    "primary_emotion": "Surprise",
    "intensity": 4,
    "valence": 2,
    "arousal": 6,
    "voice_qualities": ["Raised pitch", "Fast speech rate"],
    "authenticity": "Clearly genuine",
    "confidence": 4
  },
  "annotator": "rater_01",
  "timestamp": "2024-12-05T10:30:00Z"
}

片段級情感標註

對於包含情感變化的較長音訊：

yaml

annotation_schemes:
  - annotation_type: audio_annotation
    name: emotion_segments
    description: "Mark time segments with different emotions"
    labels:
      - name: Happy
        color: "#FCD34D"
      - name: Sad
        color: "#60A5FA"
      - name: Angry
        color: "#F87171"
      - name: Neutral
        color: "#9CA3AF"

品質控制

yaml

quality_control:
  attention_checks:
    enabled: true
    gold_items:
      - audio: "/audio/gold/clearly_happy.wav"
        expected:
          primary_emotion: "Happiness"
          intensity: [4, 5]  # Accept 4 or 5
      - audio: "/audio/gold/clearly_angry.wav"
        expected:
          primary_emotion: "Anger"

情感標註技巧

完整聆聽：始終聽完整個片段
關注聲音：情感資訊在於"怎麼說"
文化意識：不同文化的表達規範有所不同
疲勞管理：適時休息——情感標註很消耗精力
校準對齊：定期團隊討論以提高一致性

下一步

新增說話人分割用於多說話人情感追蹤
設定眾包進行大規模資料收集
計算情感任務的標註者一致性

文件請參閱音訊標註。