# EEG, ECG, and Sleep Staging Annotation with Time Series

Source: https://www.potatoannotator.com/docs/annotation-types/time-series

*New in v2.13.0*

The `time_series` display draws a multi-channel recording in the browser, and the `interval_label` scheme records labeled intervals and point events on the recording. Sleep researchers use the pair for sleep staging. Sleep staging assigns each 30-second epoch of a [polysomnogram](https://en.wikipedia.org/wiki/Polysomnography) one of the AASM stages W, N1, N2, N3, or REM. EEG researchers mark artifacts and seizures, cardiology researchers review beats and rhythms in ECG records from [PhysioNet](https://physionet.org/), wearable researchers label activity bouts in accelerometer data, and astronomers mark transits or flares in a light curve. Other questions about a recording, such as a quality rating or a note, go in ordinary schemes beside the plot.

[European Data Format](https://www.edfplus.info/) (EDF) is a standard format for exchanging and storing medical time series such as sleep and EEG recordings, and EDF+ adds annotations and recordings with gaps. [WFDB](https://wfdb.readthedocs.io/) is PhysioNet's record format. A WFDB record is a `.hea` header with `.dat` signal files and optional annotation files such as `.atr`.

- **Formats:** EDF and EDF+ (continuous and discontinuous), BDF and BDF+, WFDB records, CSV or TSV with a time column, Parquet, and a small inline JSON object. EDF and BDF files are memory-mapped and need no extra library.
- **Existing labels:** EDF+ annotations, a WFDB `.atr` file, and a [Brain Imaging Data Structure](https://bids.neuroimaging.io/) (BIDS) `events.tsv` load as pre-annotations. Annotators keep, edit, or discard the pre-annotations.
- **Long recordings:** the browser never downloads the recording. For the range on screen, the server sends at most two values per pixel per channel (the raw samples, or each pixel's minimum and maximum), and an 8-hour recording therefore opens about as fast as a 30-second one.
- **Offline:** signal files are read on the Potato server from the media directory or the task directory. URLs, absolute paths, and `..` escapes are refused.

WFDB records and Parquet files need the `signals` extra (`wfdb` and `pyarrow`):

```bash
pip install 'potato-annotation[signals]'
```

## Configuration

The config below sets up sleep staging in 30-second epochs on a four-channel polysomnogram. A second scheme marks arousals by clicking on the same plot:

```yaml
annotation_task_name: "Sleep staging"
task_dir: .
media_directory: media
output_annotation_dir: annotation_output
data_files: [data/psg.json]
item_properties: {id_key: id, text_key: note}

instance_display:
  fields:
    - key: signal               # item field holding the EDF file name
      type: time_series
      label: "PSG"
      display_options:
        window: 30
        channels:
          - {name: "EEG Fpz-Cz", range: [-120, 120]}
          - {name: "EOG", range: [-150, 150]}
          - "EMG chin"
          - "Resp"

user_config:
  allow_all_users: true

annotation_schemes:
  - annotation_type: interval_label
    name: stage
    description: "Sleep stage of each 30-second epoch"
    signal_field: signal
    mode: epoch
    epoch_length: 30
    require_all_epochs: true    # counted only while the scheme is required
    label_requirement: {required: true}
    labels:
      - {name: W, key_value: "1"}
      - {name: N1, key_value: "2"}
      - {name: N2, key_value: "3"}
      - {name: N3, key_value: "4"}
      - {name: REM, key_value: "5"}
    preannotation_labels:       # EDF+ hypnogram names -> scheme labels
      "Sleep stage W": W
      "Sleep stage 1": N1
      "Sleep stage 2": N2
      "Sleep stage 3": N3
      "Sleep stage R": REM
      "Arousal": null

  - annotation_type: interval_label
    name: arousals
    description: "Arousals (click on the plot)"
    signal_field: signal
    mode: free
    pre_annotations: false
    labels:
      - {name: arousal, key_value: "a", kind: event}
```

The `channels` list picks the lanes and their order, and a `range` fixes a lane's amplitude scale in the channel's unit. A fixed range keeps an EEG lane at the same scale in every epoch. The `preannotation_labels` map renames the EDF+ hypnogram's stage names to the scheme's labels and drops the file's arousal entries. The second scheme turns `pre_annotations` off, and annotators mark arousals there by hand. With `require_all_epochs` and `label_requirement` together, Next stays blocked until every epoch has a stage. The `signal_field` of each scheme must equal the `key` of a `time_series` field, and the server refuses to start until the two match.

Each item names its signal file relative to `media_directory`:

```json
{"id": "night-01", "note": "Night 1", "signal": "psg/night-01.edf"}
```

### Free mode for EEG review

EEG review uses free mode. In free mode, annotators drag across the plot to label an interval and click to mark a point event. The partial block below replaces `annotation_schemes` in the config above:

```yaml
annotation_schemes:
  - annotation_type: interval_label
    name: eeg_events
    description: "Mark artifacts and seizures, and click each spike"
    signal_field: signal
    mode: free
    channel_scope: either       # all channels, or the lane the drag starts on
    labels:
      - {name: artifact, key_value: "1"}
      - {name: seizure, key_value: "2"}
      - {name: spike, key_value: "3", kind: event}
```

With `channel_scope: either`, a toggle above the plot chooses whether a new interval applies to every channel or to one lane. An electrode artifact can sit on one lane while a generalized seizure spans every channel. To mark apneas or arousals inside staged epochs, keep the epoch scheme and add a free-mode scheme with the same `signal_field`, as the sleep config above does with arousals. Give the two schemes different label keys.

## Signal files

The extension of the file named in the item picks the reader:

| Extension | Format | Needs |
|---|---|---|
| `.edf` | EDF, EDF+ (continuous and discontinuous) | nothing |
| `.bdf` | BioSemi BDF, BDF+ (24-bit) | nothing |
| `.hea` | WFDB record (the header plus the `.dat` files the header lists) | `signals` extra |
| `.csv`, `.tsv` | a time column plus one column per channel | nothing |
| `.parquet`, `.pq` | the same layout, as Parquet | `signals` extra |
| `.json` | the inline object below, in a file (up to 50 MB) | nothing |

- **EDF+.** Each annotation with a duration becomes a pre-annotated interval on all channels, and each annotation without a duration becomes a point event. In EDF+D files, each data record is placed at the onset its timekeeping annotation gives, and the space between records is drawn as a break in the line. A file shorter than its header says is read up to its last complete record, with a warning under the plot.
- **WFDB.** Name the record by its `.hea` file, with the signal files in the same directory. Beat annotations in `<record>.atr` become point events labeled with their symbol (`N`, `V`, ...), and rhythm annotations such as `(AFIB` become `AFIB` intervals that run to the next rhythm change. Samples are read a window at a time, and a long Holter record is never loaded whole. Join multi-segment records before loading because the reader does not support multi-segment records.
- **CSV and Parquet.** The time column is `time_column` when set, else the first column named `time`, `t`, `timestamp`, `seconds`, `sec`, `time_s`, `datetime`, `mjd`, or `jd`, else the first column. Every other numeric column is a channel, and a unit in the header (`ax (g)`, `ay [m/s^2]`) becomes the channel's unit. A series whose time steps all fall within 0.1% of the median step is treated as regularly sampled, and any other series keeps each sample's own time. CSV and Parquet files are loaded whole and limited to 256 MB. Convert a larger recording to EDF.
- **Inline JSON.** A small signal can sit in the item itself, up to 500,000 samples across all channels.

An inline object gives a `sampling_rate` for regular samples, or a `time` list with one time per sample for an irregular series such as a light curve. Both keys may also be set per channel. `t0` sets the time of the first sample, at the top level or per channel. A `null` in `data` is a missing sample, and `start_time` gives the wall-clock time of t = 0 for `time_format: clock`:

```json
{"id": "demo", "signal": {
  "sampling_rate": 100,
  "start_time": "2024-01-01T22:00:00",
  "channels": [
    {"name": "Fz", "unit": "uV", "data": [0.1, 0.4, 0.2]},
    {"name": "Resp", "unit": "a.u.", "sampling_rate": 10, "data": [0.0, 0.3]}
  ]
}}
```

Times are seconds from the start of the recording for EDF, BDF, WFDB, and inline data with a `sampling_rate`. For CSV and Parquet, times are the values in the time column as written (seconds, days, or MJD), and a date-time column becomes seconds since the first row. Stored labels use the same time base as the file.

## Display options

Options go under `display_options` or directly on the field:

| `time_series` option | Default | Meaning |
|---|---|---|
| `window` | `30` | Seconds shown at first, and the step of the next and previous controls |
| `channels` | all | Channels to show, in order, as names or `{name, label, range: [low, high], color}` |
| `layout` | `stacked` | `stacked` gives one lane per channel, and `overlay` puts every channel on one y axis with a legend |
| `draw` | `line` | `line`, or `points` for sparse, irregular series |
| `amplitude` | `recording` | `recording` scales each lane to the channel's range over the whole file (0.5th to 99.5th percentile), and `window` rescales to what is on screen. A channel's `range` overrides both |
| `lane_height` | `56` | Pixels per lane in the stacked layout, from 20 to 400 |
| `height` | from the lanes | Plot height in pixels, from 80 to 4000. Without a value, the height is the lane count times `lane_height` plus 48, kept between 160 and 1400 |
| `time_format` | `auto` | `elapsed` (h:mm:ss), `clock` (wall time from `start_time`), or `number`. `auto` means `elapsed` when `time_unit` is `s`, and `number` otherwise |
| `time_unit` | `s` | The time column's unit, for CSV or Parquet data in days or MJD |
| `invert_y` | `false` | Flips the y axis in the overlay layout, for astronomical magnitudes |
| `time_column` | auto | The CSV or Parquet time column |
| `annotation_extension` | `atr` | The WFDB annotation file read as pre-annotations, or `null` for none |
| `max_file_mb` | `2048` | The largest signal file read, in megabytes. Larger files are refused |
| `overview` | `true` | Shows a strip under the plot with the whole recording, the labeled stretches, and the current window. Clicking or dragging the strip moves the window |

The `engine` option takes one value, `uplot`. The display refuses `allow_remote` because signal files are always read on the server.

## Scheme options

| `interval_label` option | Default | Meaning |
|---|---|---|
| `signal_field` | required | The `key` of the `time_series` field to label |
| `labels` | required | Names, or `{name, key_value, color, kind, tooltip}`. `kind` is `interval` (the default) or `event` |
| `mode` | `free` | `free` for dragging and clicking, `epoch` for labeling fixed windows in sequence |
| `epoch_length` | `30` | Seconds per epoch |
| `channel_scope` | `all` | `all` applies labels to every channel, `channel` to the lane where the drag started, and `either` shows a toggle |
| `auto_advance` | `true` | In epoch mode, moves to the next epoch after a label key |
| `allow_overlap` | `true` in free mode, `false` in epoch mode | When false, a new interval replaces the overlapped part of other intervals on the same channel |
| `next_key`, `previous_key` | `]`, `[` | Keys that move the window |
| `pre_annotations` | `true` | Starts from the file's own annotations (EDF+, WFDB `.atr`, or inline) |
| `preannotation_field` | none | An item field holding a BIDS `events.tsv` path, or annotations in the stored shape |
| `preannotation_labels` | none | Renames file labels to scheme labels, and a label mapped to `null` is dropped. A renamed annotation is kept only when the scheme declares the label with the same kind |
| `require_all_epochs` | `false` | In epoch mode, Next stays blocked until every epoch has a label. Enforced when the scheme is required |
| `min_annotations` | none | Next stays blocked until the item has at least the given number of intervals and events. Enforced when the scheme is required |
| `instructions` | none | Text shown above the labels |

Labels without a `key_value` get the keys `1` to `9`, then `q` to `p`. Label colors default to the Okabe-Ito palette. The Okabe-Ito colors stay distinguishable with the common color-vision deficiencies.

## Scoring epochs and marking intervals

In epoch mode, the recording is cut into `epoch_length` windows from its start, and annotators see one epoch at a time. A stage key labels the current epoch and moves to the next, `[` and `]` (or Page Up and Page Down) move without labeling, and "Clear this epoch" removes the label. The overview strip colors every scored epoch by its stage, and clicking the strip moves the window to the clicked point.

In free mode, annotators press a label key or click a label button, then drag across the plot to label an interval, or click to mark a point if the label is an event label. With an interval label picked, a click selects the entry under the pointer, a label key relabels the entry, Delete removes the entry, and Escape deselects. Every entry is listed under the plot with Show and Delete buttons, and the list is the keyboard route for reviewing labels without a mouse.

The A− and A+ buttons lower or raise every lane's amplitude. On a focused plot, the left and right arrow keys move one window instead of moving between items as elsewhere in Potato, and Shift with an arrow key pans by a tenth of the window.

When an item has no saved answer, the file's annotations show as dashed overlays with a note. The overlays are saved with the annotator's first change or with "Keep these", and "Discard them" clears the overlays. In epoch mode, a hypnogram stretch from the file is split into the epochs the stretch covers by at least half.

## Stored answer

Each scheme stores one JSON object per item:

```json
{"intervals": [{"start": 30.0, "end": 60.0, "label": "N2", "channel": null},
               {"start": 41.2, "end": 43.0, "label": "artifact", "channel": "EOG"}],
 "events":    [{"time": 52.25, "label": "arousal", "channel": null, "source": "file"}]}
```

`channel` is a channel name, or `null` for all channels. An entry that came from the file and was not changed keeps `"source": "file"`.

## Export

Several annotators can score the same recording, and `num_annotators_per_item` sets how many. Both signal exports keep each annotator's labels apart:

| Format | Output |
|---|---|
| `interval_label` | `intervals.csv` and `intervals.jsonl`, one row per interval or event with `user_id, instance_id, schema, kind, start, end, duration, label, channel, source`. Events have `start` equal to `end` and duration 0, and annotator entries have `source` set to `annotator` |
| `bids_events` | `<annotator>/<instance>_events.tsv` for each annotator and item, with `onset, duration, trial_type, channel` (`n/a` for all channels), plus an `events.json` sidecar. With several `interval_label` schemes, each scheme gets its own folder |

```bash
python -m potato.export -c config.yaml -f bids_events -o exports/bids
```

The generic CSV and JSONL exports also hold the answer, as one JSON cell per scheme. In epoch mode, each scored epoch is its own interval, and `intervals.csv` therefore has one row per epoch per annotator. Potato's agreement metrics do not cover `interval_label` schemes. Compute a per-epoch statistic for sleep stages, such as [Cohen's kappa](https://en.wikipedia.org/wiki/Cohen%27s_kappa), from `intervals.csv` after export.

## Example

```bash
python potato/flask_server.py start examples/advanced/time-series-intervals/config-epochs.yaml -p 8000
```

The example has two items pointing at one synthetic five-minute EDF+ polysomnogram, for scoring in 30-second epochs. The `config.yaml` beside `config-epochs.yaml` runs free mode on the same recording and on a two-minute wrist accelerometer CSV with a BIDS `events.tsv`.

## Setting up a study with a coding agent

[potato-skill](/docs/getting-started/quick-start-with-a-coding-agent) 1.2.0 targets Potato 2.13.1 and knows `interval_label` and `time_series` by name, and the skill guides the study-design side, such as what to ask annotators, how many annotators score each recording, training and attention checks, and export. The skill does not yet document the signal file formats, the display options, or the `signals` extra. Install potato-skill, describe the study to your agent, and point the agent at the Markdown copy of this page, `https://www.potatoannotator.com/docs/annotation-types/time-series.md`, for the formats and options.

## Further Reading

- [Instance Display](/docs/core-concepts/instance-display) - How display fields are configured
- [Spectrogram Annotation](/docs/annotation-types/spectrogram-annotation) - Time-frequency boxes on audio
- [Export Formats](/docs/features/export-formats) - Potato's export formats and the export command
- [How Many Annotators Do You Need?](/docs/guides/how-many-annotators) - Choosing annotators per item
- [Healthcare](/use-cases/healthcare) - Potato for clinical and biomedical annotation

For implementation details, see the [source documentation](https://github.com/davidjurgens/potato/blob/master/docs/annotation-types/science/time_series.md).