# Multirate (Matrix Rating)

Source: https://www.potatoannotator.com/docs/annotation-types/multirate

The multirate type displays multiple items in a matrix format where each item is rated on the same scale. Use it to evaluate several dimensions of a single item.

![Multi-criteria rating interface](/images/docs/likert-scale.png "Matrix rating interface for multi-dimensional evaluation in Potato")

## Basic Configuration

```yaml
annotation_schemes:
  - name: "aspect_ratings"
    description: "Rate each aspect of the response"
    annotation_type: "multirate"
    labels:
      - name: "Accuracy"
      - name: "Clarity"
      - name: "Helpfulness"
    options:
      - name: "1"
      - name: "2"
      - name: "3"
      - name: "4"
      - name: "5"
```

## Configuration Options

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `name` | string | Yes | Unique identifier for the annotation |
| `description` | string | Yes | Instructions shown to annotators |
| `annotation_type` | string | Yes | Must be `"multirate"` |
| `labels` | array | Yes | Items to be rated (rows) |
| `options` | array | Yes | Rating scale options (columns) |
| `size` | number | No | Alternative to options: number of scale points |
| `min_label` | string | No | Label for lowest rating |
| `max_label` | string | No | Label for highest rating |
| `randomize` | boolean | No | Randomize item order |
| `compact` | boolean | No | Use compact layout |

## Examples

### Response Quality Assessment

```yaml
- name: "quality_assessment"
  description: "Rate each aspect of the AI response"
  annotation_type: "multirate"
  labels:
    - name: "Accuracy"
      tooltip: "Is the information factually correct?"
    - name: "Completeness"
      tooltip: "Does it fully address the question?"
    - name: "Clarity"
      tooltip: "Is it easy to understand?"
    - name: "Relevance"
      tooltip: "Does it stay on topic?"
  size: 5
  min_label: "Poor"
  max_label: "Excellent"
```

### Translation Quality

```yaml
- name: "translation_quality"
  description: "Evaluate the translation quality"
  annotation_type: "multirate"
  labels:
    - name: "Fluency"
    - name: "Adequacy"
    - name: "Terminology"
    - name: "Style"
  options:
    - name: "1 - Unacceptable"
    - name: "2 - Poor"
    - name: "3 - Acceptable"
    - name: "4 - Good"
    - name: "5 - Excellent"
```

### Product Review Dimensions

```yaml
- name: "product_dimensions"
  description: "Rate each aspect of the product"
  annotation_type: "multirate"
  labels:
    - name: "Build Quality"
    - name: "Value for Money"
    - name: "Ease of Use"
    - name: "Customer Support"
    - name: "Documentation"
  size: 5
  min_label: "Very Poor"
  max_label: "Excellent"
  randomize: true
```

## Output Format

The multirate annotation outputs a dictionary mapping each item to its rating:

```json
{
  "id": "item_1",
  "annotations": {
    "aspect_ratings": {
      "Accuracy": "4",
      "Clarity": "5",
      "Helpfulness": "3"
    }
  }
}
```

## Use Cases

- **LLM Evaluation**: Rate responses on multiple quality dimensions
- **Translation Assessment**: Evaluate fluency, adequacy, and terminology
- **Product Reviews**: Capture ratings across different product aspects
- **Survey Research**: Likert-style matrix questions
- **Peer Review**: Rate papers on multiple criteria

## Best Practices

1. **Limit the number of items** - 3-7 items works best; more causes fatigue
2. **Use consistent scales** - All items should use the same rating scale
3. **Order items logically** - Group related dimensions together
4. **Provide clear definitions** - Use tooltips to explain each dimension
5. **Consider randomization** - Prevents order bias in responses