Docs/Annotation Types

Likert Scales

Rating scales for measuring attitudes and opinions.

Likert Scales

Likert scales allow annotators to rate items on a numeric scale, commonly used for sentiment intensity, agreement levels, and quality ratings.

Basic Configuration

annotation_schemes:
  - annotation_type: likert
    name: agreement
    description: "How much do you agree with this statement?"
    size: 5
    min_label: "Strongly Disagree"
    max_label: "Strongly Agree"

Configuration Options

Scale Size

Define the number of points:

size: 5  # 5-point scale (1-5)
size: 7  # 7-point scale (1-7)
size: 10 # 10-point scale (1-10)

Endpoint Labels

Label the scale endpoints:

min_label: "Not at all"
max_label: "Extremely"

Middle Label

For odd-numbered scales, label the midpoint:

size: 5
min_label: "Strongly Disagree"
max_label: "Strongly Agree"
mid_label: "Neutral"  # Shown at position 3

All Labels

Label every point on the scale:

size: 5
labels:
  1: "Strongly Disagree"
  2: "Disagree"
  3: "Neutral"
  4: "Agree"
  5: "Strongly Agree"

Common Scale Types

Agreement Scale (5-point)

- annotation_type: likert
  name: agreement
  size: 5
  min_label: "Strongly Disagree"
  max_label: "Strongly Agree"

Frequency Scale

- annotation_type: likert
  name: frequency
  size: 5
  labels:
    1: "Never"
    2: "Rarely"
    3: "Sometimes"
    4: "Often"
    5: "Always"

Quality Scale

- annotation_type: likert
  name: quality
  size: 5
  min_label: "Very Poor"
  max_label: "Excellent"

Importance Scale

- annotation_type: likert
  name: importance
  size: 5
  min_label: "Not Important"
  max_label: "Extremely Important"

Satisfaction Scale

- annotation_type: likert
  name: satisfaction
  size: 5
  min_label: "Very Dissatisfied"
  max_label: "Very Satisfied"

Keyboard Shortcuts

Enable number key shortcuts:

- annotation_type: likert
  name: rating
  size: 5
  keyboard_shortcuts: true  # 1-5 keys select ratings

Display Options

Horizontal Layout (Default)

- annotation_type: likert
  name: rating
  size: 5
  display: horizontal

Show Numbers

Display numeric values:

- annotation_type: likert
  name: rating
  size: 5
  show_numbers: true

Multiple Likert Scales

Rate multiple aspects:

annotation_schemes:
  - annotation_type: likert
    name: clarity
    description: "How clear is this text?"
    size: 5
    min_label: "Very Unclear"
    max_label: "Very Clear"
 
  - annotation_type: likert
    name: relevance
    description: "How relevant is this text?"
    size: 5
    min_label: "Not Relevant"
    max_label: "Highly Relevant"
 
  - annotation_type: likert
    name: quality
    description: "Overall quality rating"
    size: 5
    min_label: "Poor"
    max_label: "Excellent"

Multirate (Matrix Rating)

For rating multiple items on the same scale:

- annotation_type: multirate
  name: aspect_ratings
  description: "Rate each aspect"
  items:
    - "Clarity"
    - "Accuracy"
    - "Completeness"
    - "Relevance"
  size: 5
  min_label: "Poor"
  max_label: "Excellent"

This displays as a matrix where each row is an item and columns are scale points.

Semantic Differential

Rate between two opposite concepts:

- annotation_type: likert
  name: tone
  size: 7
  min_label: "Formal"
  max_label: "Informal"
 
- annotation_type: likert
  name: sentiment
  size: 7
  min_label: "Negative"
  max_label: "Positive"

Required vs Optional

Make the rating required:

- annotation_type: likert
  name: rating
  size: 5
  required: true

Default Value

Set a pre-selected value:

- annotation_type: likert
  name: rating
  size: 5
  default: 3  # Pre-select middle value

Full Example: Survey Task

task_name: "Content Quality Survey"
 
annotation_schemes:
  - annotation_type: likert
    name: clarity
    description: "How clear and understandable is this content?"
    size: 5
    min_label: "Very Unclear"
    max_label: "Very Clear"
    keyboard_shortcuts: true
 
  - annotation_type: likert
    name: accuracy
    description: "How accurate is the information?"
    size: 5
    min_label: "Very Inaccurate"
    max_label: "Very Accurate"
    keyboard_shortcuts: true
 
  - annotation_type: likert
    name: usefulness
    description: "How useful would this be for the target audience?"
    size: 5
    min_label: "Not Useful"
    max_label: "Very Useful"
    keyboard_shortcuts: true
 
  - annotation_type: text
    name: feedback
    description: "Any additional feedback? (Optional)"
    required: false

Best Practices

  1. Use odd numbers for scales with a neutral midpoint
  2. Use even numbers to force a direction (no neutral option)
  3. 5 or 7 points work best for most applications
  4. Label endpoints clearly to anchor the scale
  5. Be consistent across your project with scale direction
  6. Consider keyboard shortcuts for faster annotation
PreviousRadio & MultiselectNextSpan Annotation