音频标注

在 Potato 中配置基于波形的音频标注。显示交互式播放控件、设置时间标记，并与单选、李克特或文本方案组合使用。

Potato 2.0 支持音频标注，提供由 Peaks.js 驱动的波形可视化、片段标注和键盘快捷键。

用例

语音转录与审核
说话人分离
音乐分析
音频事件检测
语音情感识别
呼叫中心质量保证

启用音频支持

音频任务就是 annotation_schemes 中的一个 audio_annotation 方案。该方案从 item_properties.text_key 指定的字段读取音频 URL，因此不需要单独的媒体字段：

yaml

annotation_schemes:
  - annotation_type: audio_annotation
    name: audio_segments
    description: "Segment and label the audio"
    labels:
      - Speech
      - Music
      - Silence
      - Noise

操作模式

Potato 支持三种音频标注模式：

标签模式

分割音频并为每个片段分配类别标签：

yaml

annotation_schemes:
  - annotation_type: audio_annotation
    name: speaker_diarization
    mode: label
    description: "Identify speakers in the audio"
    labels:
      - Speaker A
      - Speaker B
      - Overlap

问题模式

每个片段拥有自己的一组问题，而不是单个标签。标注者标出一个区域，选中它，然后在片段详情面板中回答这些问题。此模式必须提供 segment_schemes，其中每一项都是一个普通的标注方案，按片段渲染：

yaml

annotation_schemes:
  - annotation_type: audio_annotation
    name: speech_quality
    mode: questions
    description: "Evaluate speech segments"
    segment_schemes:
      - annotation_type: radio
        name: speaker_type
        description: "Who is speaking?"
        labels: ["host", "guest", "unknown"]
      - annotation_type: likert
        name: clarity
        description: "How clear is this segment?"
        size: 5
        min_label: "Unintelligible"
        max_label: "Perfectly clear"

混合模式

把标签和逐片段问题结合起来。此模式需要同时提供 labels 和 segment_schemes：

yaml

annotation_schemes:
  - annotation_type: audio_annotation
    name: full_analysis
    mode: both
    description: "Label and analyze audio segments"
    labels:
      - Speech
      - Music
      - Noise
    segment_schemes:
      - annotation_type: radio
        name: foreground
        description: "Is this the foreground audio?"
        labels: ["Yes", "No"]

答案保存在片段上而不是条目上，以子方案的 name 为键。围绕片段问题做设计之前，有两个限制值得先知道：片段内部不绑定键盘快捷键，因为片段的字段只在该片段被选中时才存在；子方案上的 label_requirement 也不会被强制执行。用 min_segments 要求必须有片段，并在分析阶段检查完整性。

配置选项

基本设置

yaml

annotation_schemes:
  - annotation_type: audio_annotation
    name: segments
    description: "Create audio segments"
    labels:
      - Label A
      - Label B
 
    # Optional constraints
    min_segments: 1
    max_segments: 50

键盘快捷键

标签可以用数字键 1-9 分配：

yaml

annotation_schemes:
  - annotation_type: audio_annotation
    name: speakers
    description: "Mark each speaker turn"
    labels:
      - Speaker A  # Press 1
      - Speaker B  # Press 2
      - Overlap    # Press 3

标签颜色

自定义片段颜色：

yaml

annotation_schemes:
  - annotation_type: audio_annotation
    name: segments
    description: "Segment the recording and label each region"
    labels:
      - name: Speech
        color: "#22C55E"
      - name: Music
        color: "#3B82F6"
      - name: Silence
        color: "#9CA3AF"

波形性能

对于长音频文件，安装 BBC audiowaveform 工具可以获得最佳性能：

bash

# macOS
brew install audiowaveform
 
# Ubuntu/Debian
sudo apt-get install audiowaveform
 
# Or build from source
# https://github.com/bbc/audiowaveform

装上 audiowaveform 之后，Potato 会在服务器端预计算波形。没有它时波形由浏览器生成，这对 30 分钟以内的文件是够用的。

波形缓存

服务器端波形值得缓存：为一小时长的录音计算一次波形要花几秒钟，而标注者每次加载页面都要付出这个代价。顶层的 audio_annotation 块控制缓存：

yaml

audio_annotation:
  waveform_cache_dir: waveform_cache/     # Where pre-computed waveforms are stored
  waveform_look_ahead: 3                  # Pre-compute the next N instances
  waveform_cache_max_size: 50             # Maximum cached waveform files
  client_fallback_max_duration: 1800      # Longest file the browser will render, in seconds

只有当任务确实包含 audio_annotation 方案时，Potato 才会读取这个块。

数据格式

简单音频引用

json

[
  {"id": "1", "audio_path": "audio/recording_001.wav"},
  {"id": "2", "audio_path": "audio/recording_002.wav"}
]

yaml

data_files:
  - "data/audio_data.json"
 
item_properties:
  id_key: id
  text_key: audio_path   # The field holding the audio URL

带转录文本

json

[
  {
    "id": "1",
    "audio_path": "audio/call_001.wav",
    "transcript": "Hello, how can I help you today?"
  }
]

像这样的纯字符串会把转录文本作为上下文显示在波形旁边。如果转录文本带有时间信息，无论它来自 Whisper、云端 ASR API 还是字幕文件，都可以直接传入文件本身，Potato 会把其中的话轮渲染成与音频同步的说话人气泡：

json

[
  {
    "id": "1",
    "conversation": {"audio": "audio/call_001.wav",
                     "transcript": "audio/call_001.srt"}
  }
]

Potato 能识别 21 种转录和字幕格式，依据内容而不是扩展名来判断。参见转录格式。

输出格式

时间轴答案会被整体序列化为单个值，因此该方案在 labels 下的条目只有一个 _data 键，其值是一个 JSON 字符串。片段时间是 start_time 和 end_time，来自 segment_schemes 的逐片段答案则保存在 annotations 之下：

json

{
  "instance_id": "audio_1",
  "user_id": "annotator1",
  "labels": {
    "speaker_turns": {
      "_data": "{\"segments\": [{\"id\": \"seg_1\", \"start_time\": 0.0, \"end_time\": 2.5, \"label\": \"Speaker A\", \"annotations\": {\"clarity\": \"4\"}}, {\"id\": \"seg_2\", \"start_time\": 2.5, \"end_time\": 5.2, \"label\": \"Speaker B\", \"annotations\": {}}]}"
    }
  },
  "spans": {},
  "links": {}
}

解析之后，那个 _data 字符串是：

json

{
  "segments": [
    {
      "id": "seg_1",
      "start_time": 0.0,
      "end_time": 2.5,
      "label": "Speaker A",
      "annotations": {"clarity": "4"}
    },
    {
      "id": "seg_2",
      "start_time": 2.5,
      "end_time": 5.2,
      "label": "Speaker B",
      "annotations": {}
    }
  ]
}

视频标注的序列化方式与此相同。正是这种单键结构，使得比较两位标注者的子系统看到的是 _data 对 _data，除非它通过 server_utils/annotation_values.py 中的读取器来处理。

键盘快捷键

Potato 提供键盘快捷键以提高标注效率：

快捷键	操作
`Space`	播放/暂停
`[`	在当前位置设置片段起点
`]`	在当前位置设置片段终点
`1-9`	为当前片段分配标签
`Delete`	删除当前片段
`Left Arrow`	后退 5 秒
`Right Arrow`	前进 5 秒
`Up Arrow`	放大
`Down Arrow`	缩小
`Home`	跳到开头
`End`	跳到结尾
`+`	加快播放速度
`-`	减慢播放速度

示例配置

说话人分离

yaml

annotation_task_name: "Speaker Diarization"
task_dir: "."
port: 8000
 
data_files:
  - "data/recordings.json"
 
item_properties:
  id_key: id
  text_key: audio_path
 
annotation_schemes:
  - annotation_type: audio_annotation
    name: speakers
    mode: label
    description: "Identify who is speaking"
    labels:
      - Speaker 1
      - Speaker 2
      - Speaker 3
      - Overlap
      - Silence
    min_segments: 1
 
# Diarization recordings run long, so precompute a few ahead of the annotator.
audio_annotation:
  waveform_cache_dir: waveform_cache/
  waveform_look_ahead: 5
 
output_annotation_dir: "output/"
export_annotation_format: jsonl
 
user_config:
  allow_all_users: true

转录审核

yaml

annotation_task_name: "Transcription Quality Review"
task_dir: "."
port: 8000
 
data_files:
  - "data/transcripts.json"
 
item_properties:
  id_key: id
  text_key: audio_path
 
annotation_schemes:
  - annotation_type: audio_annotation
    name: errors
    mode: questions
    description: "Mark transcription errors"
    # Questions mode collects answers per marked region, so the error type and
    # the correction stay attached to the stretch of audio they describe.
    segment_schemes:
      - annotation_type: radio
        name: error_type
        description: "What kind of error is this?"
        labels: ["Substitution", "Deletion", "Insertion", "Speaker mix-up"]
      - annotation_type: text
        name: correction
        description: "What should the transcript say?"
 
  - annotation_type: radio
    name: overall_accuracy
    description: "Overall transcript accuracy"
    labels:
      - Accurate
      - Minor errors
      - Major errors
      - Unusable
 
output_annotation_dir: "output/"
export_annotation_format: jsonl

上面的配置对照波形逐段审核转录文本。如果转录文本出自 ASR 流程并且已经带有时间信息，speech_transcript 方案更合适。它可以直接读取 Whisper、Deepgram、AssemblyAI、SRT、WebVTT 等带时间信息的格式，并为标注者提供带错误标签和修正框的片段卡片。参见转录格式和标注 Whisper 转录文本。

呼叫中心质量保证

yaml

annotation_task_name: "Call Center Quality Assurance"
task_dir: "."
port: 8000
 
data_files:
  - "data/calls.json"
 
item_properties:
  id_key: call_id
  text_key: recording_path
 
annotation_schemes:
  # Segment-level annotation
  - annotation_type: audio_annotation
    name: conversation
    mode: both
    description: "Segment the conversation"
    labels:
      - Agent
      - Customer
      - Hold
      - Silence
    # Who is speaking is a label; how the turn went is a question about it.
    segment_schemes:
      - annotation_type: radio
        name: tone
        description: "Tone of this turn"
        labels: ["Calm", "Neutral", "Escalated"]
 
  # Call-level assessment
  - annotation_type: likert
    name: professionalism
    description: "Agent professionalism"
    size: 5
    min_label: "Poor"
    max_label: "Excellent"
 
  - annotation_type: likert
    name: resolution
    description: "Issue resolution"
    size: 5
    min_label: "Unresolved"
    max_label: "Fully resolved"
 
  - annotation_type: multiselect
    name: issues
    description: "Select any issues observed"
    labels:
      - Long hold time
      - Agent interrupted
      - Incorrect information
      - Missing greeting
      - Unprofessional language
 
  - annotation_type: text
    name: notes
    description: "Additional observations"
    rows: 4
 
output_annotation_dir: "output/"
export_annotation_format: jsonl

支持的音频格式

WAV（推荐，质量最佳）
MP3
OGG
FLAC
M4A
WebM

性能提示

安装 audiowaveform - 服务器端波形是让一小时长的文件仍然可用的关键
启用缓存 - 设置 waveform_cache_dir 来存储预生成的波形
使用 WAV 保证质量 - 压缩格式可能引入伪影
预处理音频 - 标准化音量，裁掉不必要的静音
注意文件大小 - 大文件会拖慢加载
提前预取 - 调高 waveform_look_ahead，让后面的实例在标注者到达之前就准备好

故障排除

波形未加载

检查音频文件路径是否正确
确认文件格式受支持
为长文件安装 audiowaveform
查看浏览器控制台的错误信息

性能缓慢

安装 audiowaveform 工具
用 waveform_cache_dir 启用波形缓存
减小音频文件体积
调高 waveform_look_ahead 以预计算后续实例

片段未保存

确保输出目录可写
检查标注格式配置
确认片段同时有起始和结束时间