Productivity Features

Potato includes several features to help annotators work faster and more accurately, including keyboard shortcuts, tooltips, keyword highlights, and label suggestions.

Keyboard Shortcuts

Sequential Keybindings

For annotation schemes with at most 10 options, keybindings can be assigned sequentially by default:

annotation_schemes:
  - annotation_type: radio
    name: sentiment
    labels: [positive, neutral, negative]
    sequential_key_binding: true

The first option will correspond to the "1" key, the second to "2", and so on.

Custom Keybindings

For greater control, configure custom keybindings on each label:

annotation_schemes:
  - annotation_type: multiselect
    labels:
      - name: "Option 1"
        key_value: "1"
      - name: "Option 2"
        key_value: "2"
      - name: "Skip"
        key_value: "s"

Admin Keyword Highlights

Help annotators identify relevant words and phrases with colored highlights around matching text.

Configuration

keyword_highlights_file: data/keywords.tsv

TSV File Format

The keywords file should be tab-separated with three columns:

Word	Label	Schema
love	positive	sentiment
hate	negative	sentiment
excel*	positive	sentiment
disappoint*	negative	sentiment

Matching Behavior

• Case-insensitive: "Love" matches "love", "LOVE", "Love"
• Word boundaries: "love" matches "love" but not "lovely" (unless using wildcards)
• Wildcards: Use * for prefix/suffix matching:
  • excel* matches "excellent", "excels", "excel"
  • *happy matches "unhappy", "happy"

Configuring Colors

Colors are configured in the ui.spans.span_colors section:

ui:
  spans:
    span_colors:
      sentiment:
        positive: "(34, 197, 94)"    # Green
        negative: "(239, 68, 68)"    # Red
        neutral: "(156, 163, 175)"   # Gray

Randomization Settings

For research purposes, configure keyword highlight randomization to prevent annotators from relying solely on highlights:

keyword_highlights_file: data/keywords.tsv
 
keyword_highlight_settings:
  keyword_probability: 1.0       # Show 100% of keywords (0.0-1.0)
  random_word_probability: 0.05  # Highlight 5% random words as distractors
  random_word_label: "distractor"
  random_word_schema: "keyword"

Key Features:

• Persistence: Highlighted words are cached per user+instance
• Deterministic randomization: Uses hash of username + instance_id as seed
• Behavioral tracking: Records which words were highlighted

Tooltips

Add detailed explanations for each response option:

Plaintext Tooltips

annotation_schemes:
  - annotation_type: multiselect
    name: "Question"
    labels:
      - name: "Label 1"
        tooltip: "This option means..."

HTML Tooltips

For formatted tooltips, point to an HTML file:

annotation_schemes:
  - annotation_type: multiselect
    name: "Question"
    labels:
      - name: "Label 1"
        tooltip_file: "config/tooltips/label1_tooltip.html"

Label Suggestions

Display suggestions to help annotators with two modes:

• highlight: Highlight suggested labels with color
• prefill: Automatically pre-select suggested labels

Configuration

annotation_schemes:
  - annotation_type: multiselect
    name: "sentiment"
    description: "What sentiment does the text express?"
    labels: [positive, neutral, negative]
    label_suggestions: "highlight"  # or "prefill"
 
  - annotation_type: text
    name: "explanation"
    description: "Why do you think so?"
    multiline: true
    rows: 2
    label_suggestions: "prefill"

Data Format

Include suggestions in your data items:

{
  "id": "1",
  "text": "Good Job!",
  "label_suggestions": {
    "sentiment": "positive",
    "explanation": "Because I think "
  }
}

Automatic Task Assignment

Easily assign annotation tasks to different annotators:

automatic_assignment:
  on: true
  output_filename: "task_assignment.json"
  sampling_strategy: "random"
  labels_per_instance: 10
  instance_per_annotator: 50
  test_question_per_annotator: 2

Active Learning Integration

Productivity features integrate with Active Learning to prioritize the most informative instances:

active_learning:
  enabled: true
  schema_names: ["sentiment"]
  min_annotations_per_instance: 2
  min_instances_for_training: 20
  update_frequency: 10

Best Practices

1. Use keyboard shortcuts for high-volume tasks - they significantly speed up annotation
2. Add tooltips for complex or ambiguous labels to reduce inconsistency
3. Use keyword highlights to draw attention to relevant text, but consider randomization for research validity
4. Pre-fill suggestions carefully - they can bias annotators if overused

Further Reading

• UI Configuration - Customize the interface
• Active Learning - ML-based prioritization
• Task Assignment - Configure assignment strategies

For implementation details, see the source documentation.