Writing-Process Detection

Writing-process detection reads the typing dynamics captured by Keystroke Logging and produces named flags with the evidence attached, so you can see which feature values fired each one and throw out the ones you disagree with.

It needs Potato 2.7.2 or later, and it runs only where keystroke logging is already on. Since keystroke_logging.enabled defaults to false, a project that never opted in has nothing to detect on.

Warning: Flags are evidence for human review. They are not proof of misconduct, and they must never be wired to automatic rejection, payment withholding, or participant bans. A fast, fluent typist producing a clean first draft genuinely resembles transcription. An annotator on a phone genuinely resembles someone pasting. The rules below are built to minimize those collisions, and the collisions still happen. The false positives section is part of the feature, not a disclaimer.

Three tiers

No pre-trained model ships with Potato. There is no labeled corpus in the repository, so any coefficients we shipped would arrive with a validation number we had invented. You get six rules you can read and argue with instead. If you have real labels, tier 3 is the path that reproduces the accuracies in the literature.

Tier 1: the rules

Every flag returns the feature values that fired it, so you can defend or discard any individual one.

paste_dominant

Severity: suspect. Fires when pasted_fraction >= 0.5, meaning half or more of the final text arrived by paste.

Suppressed when every paste was classified as self (re-arranging your own draft) or instance_text (quoting the passage under annotation).

silent_insertion

Severity: suspect. Fires when external_insert_ratio >= 0.3, meaning nearly a third of inserted characters appeared with no corresponding keystroke.

This is the highest-value single signal. Paste, autofill, dictation, and programmatic injection all show up here, even when the paste event itself is suppressed by the page. It is a direct operationalization of Asher et al.'s "keystroke count anomalously low relative to response length".

Suppressed on soft keyboards (virtual_keyboard) and during IME composition, where keydown is not reliably emitted and every insert would otherwise look silent. The rule uses the source-aware ratio, so legitimate quoting does not count.

transcription_rhythm

Severity: review. Fires when all three of these hold:

• iki_log_cv <= 0.06, a metronomic typing rhythm
• revision_ratio <= 0.02, essentially no deletion
• fewer than 0.5 pauses of 2 seconds or longer per 100 characters

Conjunctive by design. Any one alone has an innocent reading. Together they are the copy-typing signature Crossley et al. describe: linear, burst-oriented, low variance.

Skipped entirely for responses under 80 characters or 40 keystrokes, where there is no rhythm to speak of.

offscreen_composition

Severity: suspect. Fires when a large externally-sourced insertion (80 characters or more) immediately follows 10 seconds or more away from the page. This is the "switched to ChatGPT, came back, pasted" pattern.

Suppressed when the insertion's source was the passage or the annotator's own text, since stepping away and then quoting the passage is ordinary behavior.

implausible_speed

Severity: review. Fires above 900 characters per minute (roughly 180 wpm) sustained across a whole response.

synthetic_input

Severity: suspect. Fires on any event with isTrusted === false, the browser reporting that input was generated by script rather than by a person. Browser automation, injected scripts, and some accessibility tooling all produce this.

Verdict levels

Overriding thresholds

keystroke_logging:
  detection:
    thresholds:
      paste_dominant.pasted_fraction: 0.4
      silent_insertion.ratio: 0.25
      transcription_rhythm.iki_log_cv: 0.05
      transcription_rhythm.revision_ratio: 0.02
      transcription_rhythm.pause_2s_per_100_chars: 0.5
      offscreen_composition.blur_ms: 15000
      offscreen_composition.insert_chars: 100
      implausible_speed.chars_per_min: 1000

Thresholds are evaluated server-side and never sent to the browser. Publishing them would tell an annotator exactly how slowly to paste in order to stay under the flag.

Tier 2: project calibration

Keystroke features vary substantially by writing task (Conijn et al. 2019), and fixed thresholds on a mixture-distributed quantity are biased (Roeser et al. 2021). A cutoff that works for a one-sentence rationale is wrong for a five-paragraph essay.

Calibration sets each threshold at a tail percentile of your own project's sessions.

# Inspect the fit without saving it
python -m potato.typing_detect calibrate config.yaml --dry-run
 
# Save it
python -m potato.typing_detect calibrate config.yaml

keystroke_logging:
  detection:
    calibrate: true    # use the saved fit

Calibration requires at least 30 usable sessions (80 characters or more, non-mobile). Below that it returns insufficient_data and the defaults stay in force.

Warning: A percentile cutoff is a relative outlier definition. By construction it flags roughly tail_fraction (default 5%) of sessions even in a population where nobody is doing anything wrong. It tells you where to look first. It is not evidence. Calibrated thresholds are additionally clamped to within 3x the built-in defaults, so a homogeneous population cannot drag a cutoff onto its own median and start flagging honest annotators.

An explicit thresholds: override always beats a calibrated value, which always beats the built-in default.

Tier 3: supervised classifier

If you have labeled sessions, train a real model:

from potato import typing_store
from potato.typing_detect import fit_supervised
 
rows = typing_store.feature_matrix(task_dir, project)
labels = [...]   # 1 = non-composed, 0 = composed
 
result = fit_supervised(rows, labels, model="random_forest")
print(result["cv_accuracy_mean"], result["feature_importances"])

Requires scikit-learn, which is imported lazily and is not a Potato dependency.

Getting labels without an external study

Potato's training phase can generate them for you. Ask annotators to copy a supplied passage as a warm-up task. Their copying sessions are genuine transcription exemplars and their normal answers are composed exemplars, giving you a labeled set from inside your own project, on your own task, with your own annotators. This mirrors how Crossley et al. built their corpus, and the calibration example sets it up end to end.

False positives

The failure mode of this feature is accusing an honest annotator. These are the cases that legitimately resemble the patterns above.

The last four are real limitations rather than oversights. If your participant pool includes dictation users, assistive-technology users, or people who use writing extensions, either drop those rules from your review criteria or treat every flag as a prompt to ask the annotator rather than to act.

Accessibility

synthetic_input keys off isTrusted, which some assistive technologies also trip. Flagging a disabled annotator for using the tools they need to work is an unacceptable outcome.

That rule has no threshold, so there is no value you can set to make it unreachable. If your study is open to assistive-technology users, either exclude synthetic_input from whatever review criteria you write down, or turn detection off entirely and analyze the exported features yourself:

keystroke_logging:
  enabled: true
  detection:
    enabled: false

Threshold-based rules can be made unreachable by setting an absurd cutoff, for example implausible_speed.chars_per_min: 1000000.

Real-time intervention

keystroke_logging:
  detection:
    on_external_insert: flag   # allow | warn | block | flag

block is a blunt instrument. It also blocks legitimate quoting, and a determined participant can retype instead. Using flag and then reviewing is usually better.

Admin dashboard

The Writing Process panel under the Behavioral tab of the Admin Dashboard shows per-annotator medians, paste rates, silent-insertion rates, and every flagged session with its evidence.

It reports writing_process_risk, the share of a user's sessions that fired each flag, weighted toward the signals with the least innocent explanation. This is a ranking aid, deliberately kept separate from the existing suspicion_score so that neither number silently changes the other's meaning.

Research grounding

Every citation below is verified against the Crossref or DataCite registry.

Crossley, Tian, Choi, Holmes and Morris (2024) collected 500 argumentative essays, had other workers transcribe them, and separated authentic from transcribed at 99% accuracy with a random forest (96-98% for other models). Their feature families are what Potato captures: pause times before sentences and words, insertion and deletion counts, product-to-process ratios, bursts, revision, and process variance. Their finding is the design target. Authentic writing shows longer pauses, more insertions and deletions, and greater variance; transcription is linear and burst-oriented.

Deane, Zhang, Hao and Li, and separately Zhang, Feng, He, Li and Zhu, independently confirm the copy-typing versus natural-writing separation, the latter with a deep-learning model. Asher, Gold, Chen and Carvalho is the crowdsourcing-specific case: a keystroke tool on Prolific flagging participants who paste into response fields or whose keystroke count is anomalously low for their response length.

For the underlying process measures, see Leijten and Van Waes (Inputlog) for the standard log measures, Chenoweth and Hayes for the burst construct, Conijn, Roeser and van Zaanen for task-dependence of keystroke features, and Roeser, De Maeyer, Leijten and Van Waes for why fixed pause thresholds are biased.

References

1. Crossley, S., Tian, Y., Choi, J. S., Holmes, L., & Morris, W. (2024). Plagiarism Detection Using Keystroke Logs. EDM 2024 (Short Papers). doi:10.5281/zenodo.12729864
2. Deane, P., Zhang, M., Hao, J., & Li, C. Using Keystroke Dynamics to Detect Nonoriginal Text. Journal of Educational Measurement, 63(1). doi:10.1111/jedm.12431
3. Asher, M. W., Gold, G., Chen, E., & Carvalho, P. F. (2026). Chatbots Are Undermining Crowdsourced Research in the Behavioral Sciences: Detecting Artificial Intelligence-Assisted Cheating With a Keystroke-Based Tool. Advances in Methods and Practices in Psychological Science, 9(1). doi:10.1177/25152459261424723
4. Zhang, M., Feng, L., He, X., Li, C., & Zhu, M. (2026). Disentangling copy typing and natural writing behaviors using keystroke logs and deep learning model. Assessing Writing. doi:10.1016/j.asw.2026.101070
5. Leijten, M., & Van Waes, L. (2013). Keystroke Logging in Writing Research: Using Inputlog to Analyze and Visualize Writing Processes. Written Communication, 30(3), 358-392. doi:10.1177/0741088313491692
6. Chenoweth, N. A., & Hayes, J. R. (2001). Fluency in Writing: Generating Text in L1 and L2. Written Communication, 18(1), 80-98. doi:10.1177/0741088301018001004
7. Conijn, R., Roeser, J., & van Zaanen, M. (2019). Understanding the keystroke log: the effect of writing task on keystroke features. Reading and Writing, 32(9), 2353-2374. doi:10.1007/s11145-019-09953-8
8. Roeser, J., De Maeyer, S., Leijten, M., & Van Waes, L. (2021). Modelling typing disfluencies as finite mixture process. Reading and Writing. doi:10.1007/s11145-021-10203-z
9. Lee, M., Liang, P., & Yang, Q. (2022). CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities. CHI 2022. doi:10.1145/3491102.3502030

Further Reading

• Keystroke Logging - what is captured and how
• Keystroke Logging Ethics - IRB, consent, participant rights
• Quality Control - attention checks and gold standards
• Admin Dashboard - the Writing Process panel

For implementation details, see the source documentation.