Coming from jsPsych: Concept Map and jsPsych Tasks in Potato

jsPsych is a JavaScript library for behavioral experiments that run in a web browser, and Potato is a server for annotation and survey studies. Most jsPsych concepts have a Potato config key, and since Potato 2.12.0 a study that needs trial-level control can run its jsPsych timeline as one page of a Potato study.

A jsPsych experiment is a timeline of trials. Each trial is an object handled by a plugin, such as html-keyboard-response for a stimulus and a key press, and the timeline runs in the participant's browser. jsPsych does not host anything, so a jsPsych study also needs a server to serve the page and store the data, commonly JATOS, Pavlovia or Cognition.run. A Potato study is a sequence of phase pages, such as consent, a screener, instructions, training, annotation and a debrief, defined in a YAML config, and Potato is the server. In Potato terms, a timeline becomes phases, ending the experiment early becomes exit_when, and the condition assignment a jsPsych study gets from its host becomes conditions.

Choosing between jsPsych and Potato

jsPsych fits studies in which stimulus timing and reaction times are the data. Reaction-time paradigms such as Stroop, flanker, the implicit association test and visual search need control over what is on the screen and when, and jsPsych gives that to code that runs on every trial. jsPsych also covers eye tracking through its WebGazer extension, audio with scheduled onsets, and custom canvas drawing.

Potato fits when the study is mostly judgments on items, such as labels, ratings, rankings, spans marked in text, pairwise comparisons and questionnaires. It assigns items to participants, trains and screens them, reports inter-annotator agreement (inter-rater reliability), and handles the arrival from and return to Prolific or a SONA subject pool. The pages around a task, such as consent, screeners, instructions, practice with feedback and a post-study survey, are config entries rather than code.

A study needs both when it combines a timed task with ratings or surveys, for example a Stroop block followed by valence ratings of words. Put the jsPsych timeline on a type: jspsych page and let Potato run the rest. The trial data is stored per participant next to the ratings, and the participant gets one completion code at the end.

Concept map from jsPsych to Potato

The table pairs each jsPsych concept with the Potato key that does the same job. Most rows are a config key, and the notes say where the match is partial.

Paradigms that need a jsPsych timeline

Some paradigms need trial code of your own, and Potato's item pages do not run it. Keep these in a jsPsych timeline:

• Frame-locked presentation. Stimuli drawn on a canvas, masks timed to a frame, and audio scheduled with the Web Audio API.
• Mouse trajectories. Potato's authenticity block counts pointer moves and distance per item and keeps no positions. A jsPsych page with extensions: [mouse-tracking] records x/y samples on each trial that turns the extension on.
• Fullscreen across trials. Each Potato item is a page load, and browsers leave fullscreen when the page changes. A jsPsych page holds fullscreen for its whole timeline.
• Loops and branches inside a block of trials. Potato branches per page (show_when), per question (display_logic) and per training attempt, so a block that branches on the previous response belongs in jsPsych.

Potato's own items can also run speeded judgments. trial_timing turns each annotation item into a timed trial with a fixation cross, a stimulus duration, a response deadline and keys mapped to labels, and it measures the reaction time in the browser from the frame the stimulus appeared on. trial_timing suits lexical decision or rapid grammaticality judgments on a list of items. Onset is known to the frame (about 17 ms at 60 Hz).

Eye tracking and webcam recording are outside both kinds of page. Potato does not ship jsPsych's WebGazer extension or its calibration plugins, because WebGazer needs 'unsafe-eval' and the jsPsych page's Content-Security-Policy does not allow it. An eye-tracking study runs on a jsPsych host such as JATOS or Pavlovia, or in Gorilla.

A Potato study with a jsPsych task

The config below is the core of the Stroop example that ships with Potato. Participants are assigned to one of two feedback conditions, answer a screener that ends the study for anyone reporting difficulty telling colors apart, do a Stroop task in jsPsych, rate words, and finish with a debrief survey whose questions appear in a random order for each participant, with the comment box last. The block is partial. The full version, with the data file, survey files and timeline, is in examples/advanced/jspsych-stroop/.

conditions:
  factors:
    feedback: [none, immediate]
  assignment: balanced          # each arrival goes to the cell with the fewest participants
 
phases:
  order: [screener, stroop, annotation, debrief]
  screener:
    type: prestudy
    file: surveys/screener.json
    exit_when:
      show_when:
        - {schema: colour_vision, operator: equals, value: "Yes"}
      message: This study needs you to tell red, green and blue apart quickly, so we cannot include you.
  stroop:
    type: jspsych
    timeline: tasks/stroop.js
    plugins: [instructions, html-keyboard-response]
  debrief:
    type: poststudy
    file: surveys/debrief.json
    randomize_questions:
      pin_last: [comments]      # the open comment box stays last

Balanced assignment keeps the two feedback conditions within one participant of each other as people arrive. Adding balance_on: completed makes only participants who finish without being screened out count toward a condition, so a dropout is replaced by the next arrival.

The timeline file assigns a function to window.potatoTimeline. Potato creates the jsPsych instance, calls the function with it and a potato object, and saves the data when the timeline ends. The function returns the trials, and it can read the participant's condition to decide which ones to include:

window.potatoTimeline = function (jsPsych, potato) {
    var trial = {
        type: jsPsychHtmlKeyboardResponse,
        stimulus: '<p style="font-size:48px;color:#c62828">GREEN</p>',
        choices: ['r', 'g', 'b']
    };
    var feedback = {
        type: jsPsychHtmlKeyboardResponse,
        stimulus: '<p>Correct key: R</p>',
        trial_duration: 800
    };
    return potato.conditions.feedback === 'immediate' ? [trial, feedback] : [trial];
};

The timeline does not call initJsPsych or set an experiment-level on_finish, because Potato does both. Trial-level on_finish and the rest of the jsPsych API work as usual. potato.participant holds the Prolific or SONA ID, potato.asset(path) returns the URL of a stimulus in the page's assets_dir, and expose_answers passes answers from earlier survey pages into potato.answers.

Potato ships jsPsych 8.3.0 and 15 of its plugins, including the keyboard, button, slider, audio, video, survey, preload and fullscreen plugins. A study that uses only these plugins loads nothing from a CDN, so it runs on a server without internet access. Other plugins, including ones from jspsych-contrib, load through extra_scripts as browser builds. Start the example from the Potato repository root:

python potato/flask_server.py start examples/advanced/jspsych-stroop/config.yaml -p 8000

Trial data and export from a jsPsych page

Each run of the timeline is stored in the project database (table runtime_runs) with the participant, the page, the attempt number and the participant's conditions, and every trial row gets potato_user, potato_run_id, potato_attempt and one condition_<factor> column per factor. The jspsych export format writes one CSV of runs, one CSV of trials with every column jsPsych recorded, jsPsych's blur and fullscreen events, and each run's raw JSON:

python -m potato.export --config config.yaml --format jspsych --output ./export/

With save: incremental, each trial is stored as it finishes, and a participant who closes the tab still leaves the trials they completed. A reload starts the timeline over, because jsPsych cannot resume a timeline midway. A run that had already sent data stays on record as incomplete, and the next run carries a higher potato_attempt, so an analysis can exclude second attempts or keep them.

The CSV export of the same study writes participants.csv, one row per participant with their condition, page orders, skipped pages and any screen-out. Potato calls this the flow record, and it joins the trial data on the participant ID. jsPsych runs are stored in the task directory's project.sqlite whichever storage backend the server uses.

jsPsych's own timing applies on the page, because Potato runs nothing else there. jsPsych's timing accuracy documentation says reaction times measured in JavaScript run 10-40 ms longer than in standard lab software, with similar variance. Compare reaction times within participants, or model them with a participant term, since keyboards and displays add latency that differs between machines.

Further reading

• Behavioral Experiment Tools Compared, for jsPsych, PsychoPy, Gorilla, Qualtrics, lab.js and Potato on one page
• Multi-Phase Workflows, for consent, screeners, training and surveys
• Running a Crowdsourced Study
• Survey Instruments
• Experiment Flow, Runtime Phases and Timed Trials
• Coming from Ibex and PCIbex, for reading experiments