Keystroke Logging

Potato can record how a free-text answer was produced without recording the answer. Every event carries a timestamp, an input type, a key class and a length change; none of them carries the character that was typed. From that stream Potato computes about forty summary features and stores them with the annotation.

The point is to tell a composed answer, typed with the pauses and second thoughts of someone working it out, from a transcribed one retyped out of another window or a pasted one dropped in from a chatbot. Read the finished answers and they look alike. The logs do not.

Keystroke logging needs Potato 2.7.2 or later, and it is off by default: keystroke_logging.enabled is false until you set it, so an upgrade never starts recording anyone. For the rules built on this data, see Writing-Process Detection. Before you point it at human participants, read Keystroke Logging Ethics.

Quick start

keystroke_logging:
  enabled: true

That is the whole minimum configuration. Every free-text field in the project starts producing a content-blind event stream, a summary, and a set of detection flags.

A runnable example ships with Potato:

python potato/flask_server.py start examples/advanced/keystroke-logging/config.yaml -p 8000

Warning: enabled defaults to false. Upgrading Potato never silently starts recording your annotators.

What gets captured

Each event records a timestamp, an input type, a key class, the caret position, and the change in field length:

{t_ms: 1240, input_type: "insertText",            key_class: "letter", pos: 41, delta: +1}
{t_ms: 1310, input_type: "insertText",            key_class: "letter", pos: 42, delta: +1}
{t_ms: 3980, input_type: "deleteContentBackward", key_class: "bksp",   pos: 42, delta: -1}
{t_ms: 9120, input_type: "insertFromPaste",       key_class: "unknown",pos: 43, delta: +287,
    meta: {paste_source: "external", paste_hash: "sekqf3"}}

What is deliberately not captured

Key classes

The key itself is never stored, only which family it belongs to:

letter, digit, punct, space, enter, bksp, del, nav, mod, func, unknown

Input types

Potato's primary signal is InputEvent.inputType on beforeinput, not keydown. This is the central technical choice. Paste, drag-and-drop, IME composition, dictation, autofill, and undo all mutate a field without firing keydown at all, so a keydown-only logger is blind to precisely the cases this feature exists to detect.

Captured input types: insertText, insertReplacementText, insertFromPaste, insertFromDrop, insertCompositionText, insertLineBreak, insertParagraph, deleteContentBackward, deleteContentForward, deleteWordBackward, deleteWordForward, deleteByCut, deleteByDrag, historyUndo, historyRedo, plus the synthetic focus, blur, and keydown.

keydown and keyup are still listened to, but only to count physical keystrokes and measure dwell. The gap between characters that appeared and keys actually pressed is the strongest single signal collected. See silent_insert_ratio below.

Which fields are instrumented

By default, every free-text field: the text schema, free-response boxes inside radio and multiselect, and the rationale or notes textareas in text_edit, pairwise, trajectory_eval, and similar schemas.

Fields are identified by the schema and label_name attributes Potato already stamps on every annotation input, falling back to splitting the name attribute on :::.

Restrict the scope with either list:

keystroke_logging:
  enabled: true
  include_schemas: [rationale]      # allowlist; empty = all fields
  exclude_schemas: [scratch_notes]  # denylist

Or opt a single element out in custom HTML:

<textarea data-keystroke-logging="off" ...></textarea>

Configuration reference

keystroke_logging:
  enabled: false                # master switch
  fidelity: events              # off | summary | events
  include_schemas: []           # empty = every free-text field
  exclude_schemas: []
  store_events: true            # persist raw streams (needs fidelity: events)
  classify_paste_source: true   # label pastes self/instance_text/ai_suggestion/external
  idle_session_ms: 30000        # close a session after this much inactivity
  flush_interval_ms: 5000       # how often the browser posts completed sessions
  pause_thresholds_ms: [500, 1000, 2000, 5000, 10000]
  disclose_to_annotators: true  # show a recording notice
  detection:
    enabled: true
    calibrate: false            # use project-fitted thresholds
    on_external_insert: flag    # allow | warn | block | flag
    thresholds: {}              # per-rule overrides

Detection keys are documented in Writing-Process Detection.

Choosing a fidelity

events is the recommended setting. A 500-word response costs about 5 KB, and it means a metric you think of after data collection can still be computed.

Summary features

One summary per (user, instance, field). The feature families follow Crossley et al. (2024); see the research grounding.

Volume and product-to-process

Rhythm

Log scale because inter-key interval distributions are heavily right-skewed. Intervals above 30 seconds are excluded from these statistics so one coffee break cannot dominate them.

Pausing

Bursting

Revision

External insertion

Use external_insert_ratio for detection. silent_insert_ratio counts all silent insertion, including the legitimate kind.

Attention

Integrity

Where the data is stored

Two destinations, for two different reasons.

Raw streams go to SQLite

<task_dir>/project.sqlite, table typing_sessions, one row per session, through the same persistence layer as memos and the codebook.

Queryable summary columns are denormalized alongside a full JSON summary and a zlib-packed event blob:

sqlite3 <task_dir>/project.sqlite "
  SELECT user_id, schema_name, keystrokes, final_chars,
         pasted_fraction, silent_insert_ratio, iki_log_cv,
         json_extract(flags,'\$.level') AS level
  FROM typing_sessions;"

The stream is stored as one packed blob per session rather than one row per keystroke. It is only ever read back wholesale, and at roughly 2 bytes per event a row-per-keystroke schema would put tens of millions of rows in a project file for no query benefit.

Phase pages

Free-text answers in the training phase and in prestudy or poststudy surveys are captured too. Those pages have no instance id, so their sessions bucket under the __phase_page__ sentinel that the rest of the behavioral system already uses, and are identified by their phase and page columns instead:

SELECT phase, page, count(*) FROM typing_sessions GROUP BY phase, page;

This is what makes the calibration example work. A copy-the-passage task in the training phase yields transcription exemplars that can be told apart from ordinary composed answers by phase alone.

Summaries go to user_state.json

The compact sketch is mirrored into <output_annotation_dir>/<user>/user_state.json under instance_id_to_behavioral_data.<instance>.typing_summaries, keyed "{schema}:::{label}", so it travels with the annotation into the admin dashboard and the exports.

Raw streams deliberately do not go there. That file is fully re-serialized and atomically rewritten on every annotation save, and a long response is thousands of events.

Exporting

Both exports are opt-in, so behavioral data is never included in a dataset release by accident.

Summary features alongside annotations

export_include_typing_dynamics: true

Produces typing_dynamics.csv (or .tsv) next to annotations.csv, one row per (user, instance, field), with the summary features and the detector verdict.

Raw streams

python -m potato.export.cli <config.yaml> --format keystrokes

Writes keystroke_sessions.parquet and keystroke_events.parquet, falling back to JSONL when pyarrow is not installed. See Parquet Export for the wider exporter.

import pandas as pd
events = pd.read_parquet("keystroke_events.parquet")
 
# Inter-key intervals for one session
s = events[events.session_id == events.session_id.iloc[0]].sort_values("t_ms")
iki = s.t_ms.diff().dropna()
print(iki.median(), iki.std())
 
# Every externally-sourced paste in the project
print(events[events.paste_source == "external"])

API endpoints

Sessions are summarized server-side. The browser never sends a computed summary, so the numbers cannot be forged by a modified client, and a metric added later can be recomputed from the stored streams.

How sessions work

A session begins when a field is focused and ends at whichever comes first: losing focus, navigating to another instance, idle_session_ms of inactivity, or page unload. Completed sessions are posted every flush_interval_ms, and via navigator.sendBeacon on unload so an in-progress session is not lost.

Multiple sessions on the same field are merged before the summary is written to the user state, so leaving a field and coming back reads as one response rather than several suspiciously short ones. Counts and durations add. Distribution statistics are keystroke-weighted approximations, so use the raw streams if you need an exact pooled distribution.

Troubleshooting

No data is being recorded

Check keystroke_logging.enabled: true and that fidelity is not off. In the browser console, window.keystrokeTracker should exist with isInitialized === true. If it is undefined, the config never reached the template.

The tracker exists but no sessions appear

Check field identification:

const el = document.querySelector('textarea');
window.keystrokeTracker.getFieldIdentity(el);   // null means it is not tracked

null means the element has no schema or label_name attributes and no :::-separated name, or that it is excluded by config.

silent_insertion flags every mobile annotator

It should not, because the rule is suppressed when virtual_keyboard is true. If detection is misfiring, check that the client set that flag. See the false positives table.

project.sqlite is growing

Roughly 2 bytes per keystroke. Set fidelity: summary to keep the features and drop the streams, or use typing_store.delete_for_user() to remove one participant's data.

Numbers look wrong for automated tests

Browser automation types at near-zero intervals, which genuinely trips implausible_speed. That is the flag working rather than a bug.

Further Reading

• Writing-Process Detection - the six rules and the three detection tiers
• Keystroke Logging Ethics - consent, IRB, retention, participant rights
• Behavioral Tracking - the wider interaction-tracking system this sits inside
• Quality Control - attention checks and gold standards
• Admin Dashboard - where the Writing Process panel lives

For implementation details, see the source documentation.