音訊標註

在 Potato 中配置基於波形的音訊標註。顯示互動式播放控制元件、設定時間標記，並與單選、李克特或文本方案組合使用。

Potato 2.0 支援音訊標註，提供由 Peaks.js 驅動的波形視覺化、片段標註和鍵盤快捷鍵。

用例

語音轉錄與稽核
說話人分離
音樂分析
音訊事件檢測
語音情感識別
呼叫中心品質保證

啟用音訊支援

音訊任務就是 annotation_schemes 中的一個 audio_annotation 方案。該方案從 item_properties.text_key 指定的欄位讀取音訊 URL，因此不需要單獨的媒體欄位：

yaml

annotation_schemes:
  - annotation_type: audio_annotation
    name: audio_segments
    description: "Segment and label the audio"
    labels:
      - Speech
      - Music
      - Silence
      - Noise

操作模式

Potato 支援三種音訊標註模式：

標籤模式

分割音訊併為每個片段分配類別標籤：

yaml

annotation_schemes:
  - annotation_type: audio_annotation
    name: speaker_diarization
    mode: label
    description: "Identify speakers in the audio"
    labels:
      - Speaker A
      - Speaker B
      - Overlap

問題模式

每個片段擁有自己的一組問題，而不是單個標籤。標註者標出一個區域，選中它，然後在片段詳情面板中回答這些問題。此模式必須提供 segment_schemes，其中每一項都是一個普通的標註方案，按片段渲染：

yaml

annotation_schemes:
  - annotation_type: audio_annotation
    name: speech_quality
    mode: questions
    description: "Evaluate speech segments"
    segment_schemes:
      - annotation_type: radio
        name: speaker_type
        description: "Who is speaking?"
        labels: ["host", "guest", "unknown"]
      - annotation_type: likert
        name: clarity
        description: "How clear is this segment?"
        size: 5
        min_label: "Unintelligible"
        max_label: "Perfectly clear"

混合模式

把標籤和逐片段問題結合起來。此模式需要同時提供 labels 和 segment_schemes：

yaml

annotation_schemes:
  - annotation_type: audio_annotation
    name: full_analysis
    mode: both
    description: "Label and analyze audio segments"
    labels:
      - Speech
      - Music
      - Noise
    segment_schemes:
      - annotation_type: radio
        name: foreground
        description: "Is this the foreground audio?"
        labels: ["Yes", "No"]

答案儲存在片段上而不是條目上，以子方案的 name 為鍵。圍繞片段問題做設計之前，有兩個限制值得先知道：片段內部不繫結鍵盤快捷鍵，因為片段的欄位只在該片段被選中時才存在；子方案上的 label_requirement 也不會被強制執行。用 min_segments 要求必須有片段，並在分析階段檢查完整性。

配置選項

基本設定

yaml

annotation_schemes:
  - annotation_type: audio_annotation
    name: segments
    description: "Create audio segments"
    labels:
      - Label A
      - Label B
 
    # Optional constraints
    min_segments: 1
    max_segments: 50

鍵盤快捷鍵

標籤可以用數字鍵 1-9 分配：

yaml

annotation_schemes:
  - annotation_type: audio_annotation
    name: speakers
    description: "Mark each speaker turn"
    labels:
      - Speaker A  # Press 1
      - Speaker B  # Press 2
      - Overlap    # Press 3

標籤顏色

自定義片段顏色：

yaml

annotation_schemes:
  - annotation_type: audio_annotation
    name: segments
    description: "Segment the recording and label each region"
    labels:
      - name: Speech
        color: "#22C55E"
      - name: Music
        color: "#3B82F6"
      - name: Silence
        color: "#9CA3AF"

波形效能

對於長音訊檔案，安裝 BBC audiowaveform 工具可以獲得最佳效能：

bash

# macOS
brew install audiowaveform
 
# Ubuntu/Debian
sudo apt-get install audiowaveform
 
# Or build from source
# https://github.com/bbc/audiowaveform

裝上 audiowaveform 之後，Potato 會在伺服器端預計算波形。沒有它時波形由瀏覽器生成，這對 30 分鐘以內的檔案是夠用的。

波形快取

伺服器端波形值得快取：為一小時長的錄音計算一次波形要花幾秒鐘，而標註者每次載入頁面都要付出這個代價。頂層的 audio_annotation 塊控制快取：

yaml

audio_annotation:
  waveform_cache_dir: waveform_cache/     # Where pre-computed waveforms are stored
  waveform_look_ahead: 3                  # Pre-compute the next N instances
  waveform_cache_max_size: 50             # Maximum cached waveform files
  client_fallback_max_duration: 1800      # Longest file the browser will render, in seconds

只有當任務確實包含 audio_annotation 方案時，Potato 才會讀取這個塊。

資料格式

簡單音訊引用

json

[
  {"id": "1", "audio_path": "audio/recording_001.wav"},
  {"id": "2", "audio_path": "audio/recording_002.wav"}
]

yaml

data_files:
  - "data/audio_data.json"
 
item_properties:
  id_key: id
  text_key: audio_path   # The field holding the audio URL

帶轉錄文本

json

[
  {
    "id": "1",
    "audio_path": "audio/call_001.wav",
    "transcript": "Hello, how can I help you today?"
  }
]

像這樣的純字串會把轉錄文本作為上下文顯示在波形旁邊。如果轉錄文本帶有時間資訊，無論它來自 Whisper、雲端 ASR API 還是字幕檔案，都可以直接傳入檔案本身，Potato 會把其中的話輪渲染成與音訊同步的說話人氣泡：

json

[
  {
    "id": "1",
    "conversation": {"audio": "audio/call_001.wav",
                     "transcript": "audio/call_001.srt"}
  }
]

Potato 能識別 21 種轉錄和字幕格式，依據內容而不是副檔名來判斷。參見轉錄格式。

輸出格式

時間軸答案會被整體序列化為單個值，因此該方案在 labels 下的條目只有一個 _data 鍵，其值是一個 JSON 字串。片段時間是 start_time 和 end_time，來自 segment_schemes 的逐片段答案則儲存在 annotations 之下：

json

{
  "instance_id": "audio_1",
  "user_id": "annotator1",
  "labels": {
    "speaker_turns": {
      "_data": "{\"segments\": [{\"id\": \"seg_1\", \"start_time\": 0.0, \"end_time\": 2.5, \"label\": \"Speaker A\", \"annotations\": {\"clarity\": \"4\"}}, {\"id\": \"seg_2\", \"start_time\": 2.5, \"end_time\": 5.2, \"label\": \"Speaker B\", \"annotations\": {}}]}"
    }
  },
  "spans": {},
  "links": {}
}

解析之後，那個 _data 字串是：

json

{
  "segments": [
    {
      "id": "seg_1",
      "start_time": 0.0,
      "end_time": 2.5,
      "label": "Speaker A",
      "annotations": {"clarity": "4"}
    },
    {
      "id": "seg_2",
      "start_time": 2.5,
      "end_time": 5.2,
      "label": "Speaker B",
      "annotations": {}
    }
  ]
}

影片標註的序列化方式與此相同。正是這種單鍵結構，使得比較兩位標註者的子系統看到的是 _data 對 _data，除非它通過 server_utils/annotation_values.py 中的讀取器來處理。

鍵盤快捷鍵

Potato 提供鍵盤快捷鍵以提高標註效率：

快捷鍵	操作
`Space`	播放/暫停
`[`	在當前位置設定片段起點
`]`	在當前位置設定片段終點
`1-9`	為當前片段分配標籤
`Delete`	刪除當前片段
`Left Arrow`	後退 5 秒
`Right Arrow`	前進 5 秒
`Up Arrow`	放大
`Down Arrow`	縮小
`Home`	跳到開頭
`End`	跳到結尾
`+`	加快播放速度
`-`	減慢播放速度

示例配置

說話人分離

yaml

annotation_task_name: "Speaker Diarization"
task_dir: "."
port: 8000
 
data_files:
  - "data/recordings.json"
 
item_properties:
  id_key: id
  text_key: audio_path
 
annotation_schemes:
  - annotation_type: audio_annotation
    name: speakers
    mode: label
    description: "Identify who is speaking"
    labels:
      - Speaker 1
      - Speaker 2
      - Speaker 3
      - Overlap
      - Silence
    min_segments: 1
 
# Diarization recordings run long, so precompute a few ahead of the annotator.
audio_annotation:
  waveform_cache_dir: waveform_cache/
  waveform_look_ahead: 5
 
output_annotation_dir: "output/"
export_annotation_format: jsonl
 
user_config:
  allow_all_users: true

轉錄稽核

yaml

annotation_task_name: "Transcription Quality Review"
task_dir: "."
port: 8000
 
data_files:
  - "data/transcripts.json"
 
item_properties:
  id_key: id
  text_key: audio_path
 
annotation_schemes:
  - annotation_type: audio_annotation
    name: errors
    mode: questions
    description: "Mark transcription errors"
    # Questions mode collects answers per marked region, so the error type and
    # the correction stay attached to the stretch of audio they describe.
    segment_schemes:
      - annotation_type: radio
        name: error_type
        description: "What kind of error is this?"
        labels: ["Substitution", "Deletion", "Insertion", "Speaker mix-up"]
      - annotation_type: text
        name: correction
        description: "What should the transcript say?"
 
  - annotation_type: radio
    name: overall_accuracy
    description: "Overall transcript accuracy"
    labels:
      - Accurate
      - Minor errors
      - Major errors
      - Unusable
 
output_annotation_dir: "output/"
export_annotation_format: jsonl

上面的配置對照波形逐段稽核轉錄文本。如果轉錄文本出自 ASR 流程並且已經帶有時間資訊，speech_transcript 方案更合適。它可以直接讀取 Whisper、Deepgram、AssemblyAI、SRT、WebVTT 等帶時間資訊的格式，併為標註者提供帶錯誤標籤和修正框的片段卡片。參見轉錄格式和標註 Whisper 轉錄文本。

呼叫中心品質保證

yaml

annotation_task_name: "Call Center Quality Assurance"
task_dir: "."
port: 8000
 
data_files:
  - "data/calls.json"
 
item_properties:
  id_key: call_id
  text_key: recording_path
 
annotation_schemes:
  # Segment-level annotation
  - annotation_type: audio_annotation
    name: conversation
    mode: both
    description: "Segment the conversation"
    labels:
      - Agent
      - Customer
      - Hold
      - Silence
    # Who is speaking is a label; how the turn went is a question about it.
    segment_schemes:
      - annotation_type: radio
        name: tone
        description: "Tone of this turn"
        labels: ["Calm", "Neutral", "Escalated"]
 
  # Call-level assessment
  - annotation_type: likert
    name: professionalism
    description: "Agent professionalism"
    size: 5
    min_label: "Poor"
    max_label: "Excellent"
 
  - annotation_type: likert
    name: resolution
    description: "Issue resolution"
    size: 5
    min_label: "Unresolved"
    max_label: "Fully resolved"
 
  - annotation_type: multiselect
    name: issues
    description: "Select any issues observed"
    labels:
      - Long hold time
      - Agent interrupted
      - Incorrect information
      - Missing greeting
      - Unprofessional language
 
  - annotation_type: text
    name: notes
    description: "Additional observations"
    rows: 4
 
output_annotation_dir: "output/"
export_annotation_format: jsonl

支援的音訊格式

WAV（推薦，品質最佳）
MP3
OGG
FLAC
M4A
WebM

效能提示

安裝 audiowaveform - 伺服器端波形是讓一小時長的檔案仍然可用的關鍵
啟用快取 - 設定 waveform_cache_dir 來儲存預生成的波形
使用 WAV 保證品質 - 壓縮格式可能引入偽影
預處理音訊 - 標準化音量，裁掉不必要的靜音
注意檔案大小 - 大檔案會拖慢載入
提前預取 - 調高 waveform_look_ahead，讓後面的實例在標註者到達之前就準備好

故障排除

波形未載入

檢查音訊檔案路徑是否正確
確認檔案格式受支援
為長檔案安裝 audiowaveform
檢視瀏覽器控制台的錯誤資訊

效能緩慢

安裝 audiowaveform 工具
用 waveform_cache_dir 啟用波形快取
減小音訊檔案體積
調高 waveform_look_ahead 以預計算後續實例

片段未儲存

確保輸出目錄可寫
檢查標註格式配置
確認片段同時有起始和結束時間