# Annotation Schemes

Source: https://www.potatoannotator.com/docs/core-concepts/annotation-schemes

Annotation schemes define the labeling tasks for your annotators. Potato supports over 30 annotation types that can be combined to create complex annotation tasks.

## Basic Structure

Each scheme is defined in the `annotation_schemes` array:

```yaml
annotation_schemes:
  - annotation_type: radio
    name: sentiment
    description: "What is the sentiment?"
    labels:
      - Positive
      - Negative
      - Neutral
```

## Required Fields

| Field | Description |
|-------|-------------|
| `annotation_type` | Type of annotation (radio, multiselect, likert, span, text, number, slider, multirate, select, pairwise, best_worst, soft_label, confidence_annotation, constant_sum, range_slider, semantic_differential, hierarchical_multiselect, card_sort, rubric_eval, extractive_qa, error_span, triage, coreference, span_link, entity_linking, and more) |
| `name` | Internal identifier (no spaces, used in output) |
| `description` | Instructions shown to annotators |

## Supported Annotation Types

### 1. Radio (Single Choice)

Select exactly one option from a list:

```yaml
- annotation_type: radio
  name: sentiment
  description: "What is the sentiment of this text?"
  labels:
    - Positive
    - Negative
    - Neutral

  # Optional features
  keyboard_shortcuts:
    Positive: "1"
    Negative: "2"
    Neutral: "3"

  # Or use sequential binding (1, 2, 3... automatically)
  sequential_key_binding: true

  # Horizontal layout instead of vertical
  horizontal: true
```

### 2. Likert Scale

Rating scales with labeled endpoints:

```yaml
- annotation_type: likert
  name: agreement
  description: "How much do you agree with this statement?"
  size: 5  # Number of scale points
  min_label: "Strongly Disagree"
  max_label: "Strongly Agree"

  # Optional mid-point label
  mid_label: "Neutral"

  # Show numeric values
  show_numbers: true
```

### 3. Multiselect (Multiple Choice)

Select multiple options from a list:

```yaml
- annotation_type: multiselect
  name: topics
  description: "Select all relevant topics"
  labels:
    - Politics
    - Technology
    - Sports
    - Entertainment
    - Science

  # Selection constraints
  min_selections: 1
  max_selections: 3

  # Allow free text response
  free_response: true
  free_response_label: "Other (specify)"
```

### 4. Span Annotation

Highlight and label text segments:

```yaml
- annotation_type: span
  name: entities
  description: "Highlight named entities in the text"
  labels:
    - PERSON
    - ORGANIZATION
    - LOCATION
    - DATE

  # Visual customization
  label_colors:
    PERSON: "#3b82f6"
    ORGANIZATION: "#10b981"
    LOCATION: "#f59e0b"
    DATE: "#8b5cf6"

  # Allow overlapping spans
  allow_overlapping: false

  # Keyboard shortcuts for labels
  sequential_key_binding: true
```

### 5. Slider

Continuous numerical range:

```yaml
- annotation_type: slider
  name: confidence
  description: "How confident are you in your answer?"
  min: 0
  max: 100
  step: 1
  default: 50

  # Endpoint labels
  min_label: "Not confident"
  max_label: "Very confident"

  # Show current value
  show_value: true
```

### 6. Text Input

Free-form text responses:

```yaml
- annotation_type: text
  name: explanation
  description: "Explain your reasoning"

  # Multi-line input
  textarea: true

  # Character limits
  min_length: 10
  max_length: 500

  # Placeholder text
  placeholder: "Enter your explanation here..."

  # Disable paste (for transcription tasks)
  disable_paste: true
```

### 7. Number Input

Numerical input with constraints:

```yaml
- annotation_type: number
  name: count
  description: "How many entities are mentioned?"
  min: 0
  max: 100
  step: 1
  default: 0
```

### 8. Multirate (Matrix Rating)

Rate multiple items on the same scale:

```yaml
- annotation_type: multirate
  name: quality_aspects
  description: "Rate each aspect of the response"
  items:
    - Accuracy
    - Clarity
    - Completeness
    - Relevance
  size: 5  # Scale points
  min_label: "Poor"
  max_label: "Excellent"

  # Randomize item order
  randomize: true

  # Layout options
  compact: false
```

## Common Options

### Keyboard Shortcuts

Speed up annotation with keyboard bindings:

```yaml
# Manual shortcuts
keyboard_shortcuts:
  Positive: "1"
  Negative: "2"
  Neutral: "3"

# Or automatic sequential binding
sequential_key_binding: true  # Assigns 1, 2, 3...
```

### Tooltips

Provide hover hints for labels:

```yaml
tooltips:
  Positive: "Expresses happiness, approval, or satisfaction"
  Negative: "Expresses sadness, anger, or disappointment"
  Neutral: "No clear emotional content"
```

### Label Colors

Custom colors for visual distinction:

```yaml
label_colors:
  PERSON: "#3b82f6"
  LOCATION: "#10b981"
  ORGANIZATION: "#f59e0b"
```

### Required Fields

Make a scheme required before submission:

```yaml
- annotation_type: radio
  name: sentiment
  required: true
  labels:
    - Positive
    - Negative
```

## Multiple Schemes

Combine multiple annotation types per instance:

```yaml
annotation_schemes:
  # Primary classification
  - annotation_type: radio
    name: sentiment
    description: "Overall sentiment"
    labels:
      - Positive
      - Negative
      - Neutral
    required: true
    sequential_key_binding: true

  # Confidence rating
  - annotation_type: likert
    name: confidence
    description: "How confident are you?"
    size: 5
    min_label: "Guessing"
    max_label: "Certain"

  # Topic tags
  - annotation_type: multiselect
    name: topics
    description: "Select all relevant topics"
    labels:
      - Politics
      - Technology
      - Sports
      - Entertainment
    free_response: true

  # Notes
  - annotation_type: text
    name: notes
    description: "Any additional observations?"
    textarea: true
    required: false
```

## Advanced Features

### Pairwise Comparison

Compare two items:

```yaml
- annotation_type: pairwise
  name: preference
  description: "Which response is better?"
  options:
    - label: "Response A"
      value: "A"
    - label: "Response B"
      value: "B"
    - label: "Equal"
      value: "tie"

  # Allow tie selection
  allow_tie: true
```

### Best-Worst Scaling

Rank items by selecting best and worst:

```yaml
- annotation_type: best_worst
  name: ranking
  description: "Select the best and worst items"
  # Items come from the data file
```

### Dropdown Select

Space-efficient single selection:

```yaml
- annotation_type: select
  name: category
  description: "Select a category"
  labels:
    - Category A
    - Category B
    - Category C
    - Category D
    - Category E

  # Default selection
  default: "Category A"
```

## Data Format Reference

### Input

Annotation schemes work with your data format:

```json
{
  "id": "doc_1",
  "text": "This is the text to annotate."
}
```

### Output

Annotations are saved with scheme names as keys:

```json
{
  "id": "doc_1",
  "annotations": {
    "sentiment": "Positive",
    "confidence": 4,
    "topics": ["Technology", "Science"],
    "entities": [
      {"start": 0, "end": 4, "label": "ORGANIZATION", "text": "This"}
    ],
    "notes": "Clear positive sentiment about technology."
  }
}
```

## Best Practices

### 1. Clear Labels

Use unambiguous, distinct labels:

```yaml
# Good
labels:
  - Strongly Positive
  - Somewhat Positive
  - Neutral
  - Somewhat Negative
  - Strongly Negative

# Avoid
labels:
  - Good
  - OK
  - Fine
  - Acceptable
```

### 2. Helpful Tooltips

Add tooltips for nuanced labels:

```yaml
tooltips:
  Sarcasm: "The text says the opposite of what it means"
  Irony: "A mismatch between expectation and reality"
```

### 3. Keyboard Shortcuts

Enable shortcuts for high-volume tasks:

```yaml
sequential_key_binding: true
```

### 4. Logical Order

Order labels consistently:

- Most common first
- Alphabetically
- By intensity (low to high)

### 5. Limit Options

Too many choices slow annotation:

- Radio: 2-7 options
- Multiselect: 5-15 options
- Likert: 5-7 points

### 6. Test First

Annotate several examples yourself before deploying to catch:

- Ambiguous labels
- Missing categories
- Unclear instructions