Potato runs an online behavioral experiment from one YAML file, with between-subjects conditions, Latin-square counterbalancing, timed trials with reaction times, self-paced reading, and the hand-off to and from Prolific or SONA. Potato is free and open source and runs on your own machine or your institution's server, and a jsPsych timeline can run as one page of the study.
Besides the task, an online experiment needs a server to host the pages and store the data, a way to assign each participant to a condition, consent and screening pages, and a return link to the recruitment platform. jsPsych and PsychoPy build the task and leave hosting to Pavlovia, JATOS or Cognition.run, and hosted builders such as Gorilla run everything on their own servers. Potato runs the whole study on a server you control, which some ethics boards and data-protection rules require, and records every assignment and timing in the export.
Behavioral experiment designs in Potato
Each row is a setting or page type that ships with Potato 2.12, and the linked page has its options.
| Task | How Potato does it | Docs |
|---|---|---|
| Between-subjects designs | conditions assigns each participant one cell, balanced, random, or round-robin, and keeps it across reloads | Experiment flow |
| Counterbalancing | phases.counterbalance orders blocks per participant, at random or by Latin square | Experiment flow |
| Screeners and branching | show_when and exit_when on pages, and display_logic on questions | Experiment flow |
| Speeded judgments | trial_timing with fixation, stimulus duration, deadline, keys mapped to responses, and reaction times | Timed trials |
| Self-paced reading and maze | Moving-window, cumulative, and centered presentation, with a time for each region | Self-paced reading |
| jsPsych tasks | A jsPsych 8 timeline as one page, with Potato serving it and storing its data | Runtime phases |
| Questionnaires and scales | Validated instruments, Likert and slider items, and imports from Qualtrics or REDCap | Survey instruments |
| Incentive-compatible tasks | Price lists, BDM, and belief elicitation, with bonuses computed for Prolific | Incentive payments |
A two-condition study with a jsPsych task
The part of a config below assigns participants to one of two feedback conditions, balanced across the study, and runs a jsPsych Stroop timeline between consent and the main task. Online behavioral experiments builds the whole study.
conditions:
factors:
feedback: [none, immediate]
assignment: balanced
phases:
order: [consent, stroop, annotation, debrief]
stroop:
type: jspsych
timeline: tasks/stroop.js
plugins: [instructions, html-keyboard-response]assignment: balanced gives each new participant the cell with the fewest participants so far, which keeps the cells even when people drop out at different rates. Potato ships jsPsych 8.3.0 and 15 of its plugins, so a study that uses only those loads nothing from a CDN.
Recruiting through Prolific or SONA
Participants arrive from Prolific with their PROLIFIC_PID in the study URL and are logged in without a password. At the end, participants see the completion code and a button back to Prolific, and a participant who fails the attention checks or the training phase can get a separate code. Crowdsourcing has the settings for Prolific and SONA.
Data and exports
Each participant's answers, assigned condition, page order, and trial timings are stored on your server. The CSV export writes participants.csv with conditions, orders and screen-outs, trials.csv with one row per timed trial including timeouts, and regions.csv for self-paced reading. The data_package export adds a Frictionless Data Package description, and psych_ds writes the same tables in the Psych-DS layout for sharing.
Frequently asked questions
Can I host a jsPsych experiment on my own server instead of Pavlovia?
Yes. Add a page with type: jspsych that names your timeline file, and Potato serves the page, passes the timeline the participant's ID and condition, stores every trial, and moves the participant on when the timeline ends. Consent, surveys, the Prolific return, and the export stay in Potato. Potato runs on any machine where you can install Python, including a university server.
How do I send Prolific participants back with a completion code?
Set the login to read PROLIFIC_PID from the URL and put your completion code in the config. At the end of the study, participants see the code and a button back to Prolific. The prolific provider adds a second code for participants who fail the attention checks or the training phase.
Is reaction time measured in the browser accurate enough?
For most within-participant comparisons, yes. Potato reveals the stimulus inside a requestAnimationFrame callback and takes onset from the next frame, so onset is known to the frame, about 17 ms at 60 Hz. Keyboards and displays add 5 to 30 ms of latency the browser cannot see, and the latency differs between machines, so compare reaction times within participants or model a participant term. For millisecond-level work or audio onsets, run the task on a jsPsych page.
Can I run a self-paced reading experiment online without writing JavaScript?
Yes. A presentation block under trial_timing shows each item's text one region at a time, in moving-window, cumulative, or centered mode, and records how long the participant spends on each region. The maze task works the same way, and comprehension questions follow as ordinary questions. Self-paced reading gives the configuration.
Further reading
- Online behavioral experiments, for a complete study end to end
- Behavioral experiment tools compared, for what each tool is built for
- Coming from Ibex, for moving a PCIbex study across
- Survey research and linguistics, for neighboring fields on this site