Skip to content

错误片段

在 Potato 中搭建 MQM 风格的错误标注界面，用于翻译质量评估、文本纠错，以及带严重程度打分的分类错误片段标注。

错误片段标注方案提供 MQM 风格（Multidimensional Quality Metrics，多维质量指标）的界面，用带类别和严重程度的方式标出文本中的错误。它适合翻译质量评估、文本编辑审校、内容质量检查，以及任何需要细粒度错误标注的任务。

Potato error span interface for marking translation errors with severity scoresError span in Potato

概览

错误片段方案提供：

  • 分类的错误类型，可选配子类型做更细的划分
  • 严重程度等级，每一级的扣分可配置
  • 实时质量分，随着标出的错误增加而下降
  • 颜色编码的片段，在视觉上区分错误类型和严重程度

标注者选中一段文本，指定错误类型和严重程度，系统自动算出质量分。

快速开始

yaml
annotation_schemes:
  - annotation_type: error_span
    name: translation_errors
    description: Mark all errors in the translation below.
    error_types:
      - name: Accuracy
      - name: Fluency
      - name: Terminology
    show_score: true
    max_score: 100

配置项

字段类型默认值说明
annotation_typestring必填必须为 "error_span"
namestring必填本方案的唯一标识符
descriptionstring必填展示给标注者的说明文字
error_typesarray必填错误类型对象列表，每项包含 name 和可选的 subtypes 数组
severitiesarray[{name: "Minor", weight: -1}, {name: "Major", weight: -5}, {name: "Critical", weight: -10}]严重程度列表，每项包含 name 和 weight（扣分值）
show_scorebooleantrue是否显示实时质量分
max_scoreinteger100扣分之前的初始质量分

示例

翻译质量（MQM）

yaml
annotation_schemes:
  - annotation_type: error_span
    name: mqm_errors
    description: >
      Mark all errors in the machine translation.
      Select the error span, choose a category and severity.
    error_types:
      - name: Accuracy
        subtypes:
          - Mistranslation
          - Addition
          - Omission
          - Untranslated
      - name: Fluency
        subtypes:
          - Grammar
          - Spelling
          - Punctuation
          - Register
      - name: Terminology
        subtypes:
          - Inconsistent
          - Wrong Term
      - name: Style
    severities:
      - name: Minor
        weight: -1
      - name: Major
        weight: -5
      - name: Critical
        weight: -10
    show_score: true
    max_score: 100

内容编辑审校

yaml
annotation_schemes:
  - annotation_type: error_span
    name: editing_errors
    description: Mark all issues that need editing in this article.
    error_types:
      - name: Factual Error
      - name: Grammar
        subtypes:
          - Subject-Verb Agreement
          - Tense
          - Pronoun Reference
      - name: Style
        subtypes:
          - Wordiness
          - Passive Voice
          - Jargon
      - name: Formatting
    severities:
      - name: Suggestion
        weight: -1
      - name: Required Fix
        weight: -5
    show_score: false

代码评审标注

yaml
annotation_schemes:
  - annotation_type: error_span
    name: code_errors
    description: Mark issues in this code snippet.
    error_types:
      - name: Bug
        subtypes:
          - Logic Error
          - Off-by-One
          - Null Reference
      - name: Style
        subtypes:
          - Naming
          - Formatting
      - name: Security
        subtypes:
          - Injection
          - Exposure
      - name: Performance
    severities:
      - name: Nitpick
        weight: -1
      - name: Warning
        weight: -3
      - name: Blocker
        weight: -10
    max_score: 100
    show_score: true

输出格式

json
{
  "translation_errors": {
    "labels": {
      "errors": [
        {
          "start": 12,
          "end": 25,
          "text": "incorrectly translated",
          "error_type": "Accuracy",
          "subtype": "Mistranslation",
          "severity": "Major"
        },
        {
          "start": 45,
          "end": 52,
          "text": "the the",
          "error_type": "Fluency",
          "subtype": "Grammar",
          "severity": "Minor"
        }
      ],
      "score": 94
    }
  }
}

分数等于 max_score 加上所有严重程度权重之和。

实践建议

  1. 把错误类型的边界划清楚 —— 标注者不应该在两个类型之间反复纠结；在说明文字里给出例子
  2. 用子类型做细分 —— 顶层类型保持界面简洁，需要细致分析时再看子类型
  3. 认真校准严重程度权重 —— 权重比例要反映实际影响，严重错误的代价应该明显高于轻微错误
  4. 根据文本长度设置 max_score —— 文本较短时，把 max_score 调低，避免单个错误影响过大
  5. 提供标注指南 —— MQM 风格的标注很依赖详细指南，每种错误类型和严重程度都要配例子

延伸阅读

有关实现详情，请参阅源文档。