Skip to content
指南閱讀約需 4 分鐘

理解 Potato 資料格式

深入介紹 JSON 和 JSONL 資料格式，包含文本、影像、音訊和多模態標註的示例。

Potato Team

Potato 使用 JSON 和 JSONL 格式處理輸入資料和輸出標註。本指南涵蓋所有資料類型的格式規範、示例和最佳實踐。

輸入資料格式

JSON Lines (JSONL) - 推薦

每行一個 JSON 物件：

json
{"id": "001", "text": "First document text here."}
{"id": "002", "text": "Second document text here."}
{"id": "003", "text": "Third document text here."}

優勢：

  • 流式處理（記憶體高效）
  • 便於追加資料
  • 單行損壞不會影響整個檔案

JSON 陣列

標準 JSON 陣列：

json
[
  {"id": "001", "text": "First document."},
  {"id": "002", "text": "Second document."},
  {"id": "003", "text": "Third document."}
]

配置：

yaml
data_files:
  - data/items.json

文本標註資料

基本文本

json
{"id": "doc_001", "text": "The product quality exceeded my expectations."}

帶後設資料

json
{
  "id": "review_001",
  "text": "Great product, fast shipping!",
  "metadata": {
    "source": "amazon",
    "date": "2024-01-15",
    "author": "user123",
    "rating": 5
  }
}

帶預標註

json
{
  "id": "ner_001",
  "text": "Apple announced new products in Cupertino.",
  "pre_annotations": {
    "entities": [
      {"start": 0, "end": 5, "label": "ORG", "text": "Apple"},
      {"start": 31, "end": 40, "label": "LOC", "text": "Cupertino"}
    ]
  }
}

配置：

yaml
data_files:
  - data/texts.json
 
item_properties:
  id_key: id
  text_key: text

影像標註資料

本地影像

json
{
  "id": "img_001",
  "image_path": "/data/images/photo_001.jpg",
  "caption": "Street scene in Paris"
}

遠端影像

json
{
  "id": "img_002",
  "image_url": "https://example.com/images/photo.jpg"
}

帶邊界框

json
{
  "id": "detection_001",
  "image_path": "/images/street.jpg",
  "pre_annotations": {
    "objects": [
      {"bbox": [100, 150, 200, 300], "label": "person"},
      {"bbox": [350, 200, 450, 280], "label": "car"}
    ]
  }
}

配置：

yaml
data_files:
  - data/images.json
 
item_properties:
  id_key: id
  image_key: image_path  # or image_url

音訊標註資料

本地音訊

json
{
  "id": "audio_001",
  "audio_path": "/data/audio/recording.wav",
  "duration": 45.5,
  "transcript": "Hello, how are you today?"
}

帶分段

json
{
  "id": "audio_002",
  "audio_path": "/audio/meeting.mp3",
  "segments": [
    {"start": 0.0, "end": 5.5, "speaker": "Speaker1"},
    {"start": 5.5, "end": 12.0, "speaker": "Speaker2"}
  ]
}

配置：

yaml
data_files:
  - data/audio.json
 
item_properties:
  id_key: id
  text_key: transcript
 
instance_display:
  fields:
    - key: audio_path
      type: audio
      display_options:
        controls: true

多模態資料

文本 + 影像

json
{
  "id": "mm_001",
  "text": "What is shown in this image?",
  "image_path": "/images/scene.jpg"
}

文本 + 音訊

json
{
  "id": "mm_002",
  "text": "Transcribe this audio:",
  "audio_path": "/audio/clip.wav",
  "reference_transcript": "Expected transcription here"
}

輸出標註格式

基本輸出

json
{
  "instance_id": "doc_001",
  "user_id": "user123",
  "labels": {
    "sentiment": {"Positive": "true"},
    "confidence": {"5": "true"}
  },
  "spans": {},
  "links": {}
}

Span 標註

跨度按標註方案名稱分組。每條跨度存的是偏移量，標籤放在 name 裡，jsonl 匯出器還會補上這些偏移覆蓋到的 text：

json
{
  "instance_id": "ner_001",
  "user_id": "user123",
  "labels": {},
  "spans": {
    "entities": [
      {"schema": "entities", "name": "ORG", "start": 0, "end": 5, "text": "Apple"},
      {"schema": "entities", "name": "PERSON", "start": 10, "end": 18, "text": "Tim Cook"},
      {"schema": "entities", "name": "LOC", "start": 27, "end": 32, "text": "Paris"}
    ]
  },
  "links": {}
}

多標註者

匯出是每個標註者、每個條目一條記錄，所以同一條目上的兩個標註者是兩行，而不是一條巢狀記錄。Potato 不寫任何聚合欄位；多數投票和一致性都要你自己從這些行裡算：

json
{"instance_id": "item_001", "user_id": "ann1", "labels": {"sentiment": {"Positive": "true"}}, "spans": {}, "links": {}}
{"instance_id": "item_001", "user_id": "ann2", "labels": {"sentiment": {"Positive": "true"}}, "spans": {}, "links": {}}

配置參考

yaml
data_files:
  - data/items.json
 
item_properties:
  id_key: id
  text_key: text
  image_key: image_path
 
instance_display:
  fields:
    - key: audio_path
      type: audio

最佳實踐

  1. 始終包含 ID：用於追蹤的唯一識別符號
  2. 大數據集使用 JSONL：更好的記憶體效率
  3. 載入前驗證：檢查 JSON 語法
  4. 包含後設資料：來源、日期、作者有助於除錯
  5. 一致的欄位名：方便下游處理

完整的資料格式文件請參閱 資料格式。