# Run a Stroop Task on Your Own Server With jsPsych
Source: https://www.potatoannotator.com/blog/stroop-task-jspsych-own-server
Cognitive psychologists who write tasks in jsPsych can follow this tutorial to run a [Stroop task](https://en.wikipedia.org/wiki/Stroop_effect) on a server under the lab's control, with Potato 2.13.1 serving the pages and storing the data. The study has a consent page, a color-vision screener that ends the session for anyone who has trouble telling red, green, and blue apart, the jsPsych timeline with trial-by-trial feedback for half of the participants, two ratings of the task, and a debrief. Participants arrive from [Prolific](https://www.prolific.com/) and leave with a completion code, and the trial data goes into R. The configs, commands, and export outputs on this page come from runs of Potato 2.12.1 with simulated participants in a headless browser.
## Hosting options for a jsPsych study
jsPsych runs in the participant's browser and hosts nothing itself. A jsPsych study needs a server to deliver the page and receive the data. PsychoPy's [pricing page](https://psychopy.org/pricing.html), updated October 21, 2026, lists [Pavlovia](https://pavlovia.org/) credits at £0.26 per participant and a university license at £2,000 a year. [Gorilla](https://gorilla.sc/) hosts the whole study, and its [pricing FAQ](https://support.gorilla.sc/support/pricing-subscriptions-services/pricing-faq) lists academic pay-as-you-go tokens at £1.09 per participant, with a minimum purchase of 25 tokens. Both services keep the data on their own servers. In the PsychoPy forum thread ["Running Experiments on Own Server"](https://discourse.psychopy.org/t/running-experiments-on-own-server/17052), PsychoPy staff wrote in 2021 that a local install of Pavlovia was available to institutions with site licenses.
[JATOS](https://www.jatos.org/) is free and open source under the Apache 2.0 license, installs on your own server, and runs jsPsych, lab.js, and OpenSesame studies. JATOS is the established choice when the study is the jsPsych timeline alone. Potato fits when the timeline is one part of a session that also needs consent and screening pages, balanced assignment to conditions, ratings or questionnaires, and the return to Prolific. Potato sets up each of these parts in a YAML config rather than in JavaScript. [Behavioral Experiment Tools Compared](/docs/guides/behavioral-experiment-tools-compared) sets the tools side by side.
## The study
The session runs in this order:
1. **Consent.** A page describes the study and asks for agreement.
2. **Screener.** A participant who reports difficulty telling red, green, or blue apart is screened out and sent back to Prolific with a screen-out code.
3. **Stroop task.** A jsPsych timeline shows each color word in each ink color twice, 18 trials in all. Participants in the `immediate` condition see "Correct", "Incorrect", or "Too slow" after each response, and participants in the `none` condition see no feedback.
4. **Task ratings.** Two five-point ratings ask how difficult the task was and how well the participant did.
5. **Debrief.** A page explains the Stroop effect and the feedback manipulation and offers a comment box.
Feedback is a between-subjects factor, and each participant keeps one condition for the whole session. Potato assigns the condition on arrival and passes the condition to the timeline.
## Installing Potato and laying out the project
Install the release this tutorial was checked against:
```bash
pip install potato-annotation==2.13.1
```
The project folder contains the config, three survey pages, the timeline, and one data file:
```text
stroop-study/
├── config.yaml
├── data/task.jsonl
├── surveys/consent.json
├── surveys/screener.json
├── surveys/debrief.json
└── tasks/stroop.js
```
## The config
The config below is complete and passes `python -m potato.validate_cli config.yaml --strict` on 2.12.1 and on 2.13.1. The validator also checks that `tasks/stroop.js` exists.
```yaml
annotation_task_name: Stroop feedback study
task_dir: .
output_annotation_dir: annotation_output/
data_files:
- data/task.jsonl
item_properties:
id_key: id
text_key: text
login:
type: url_direct
url_argument: PROLIFIC_PID
user_config:
allow_all_users: true
users: []
crowdsourcing:
provider: prolific
prolific:
completion:
code: C1STROOP
screened_out_code: C1SCREEN
conditions:
factors:
feedback: [none, immediate]
assignment: balanced
balance_on: completed
phases:
order: [consent, screener, stroop, annotation, debrief]
consent:
type: consent
file: surveys/consent.json
screener:
type: prestudy
file: surveys/screener.json
exit_when:
show_when:
- {schema: color_vision, operator: equals, value: "Yes"}
stroop:
type: jspsych
timeline: tasks/stroop.js
plugins: [instructions, html-keyboard-response]
save: incremental
annotation:
type: annotation
debrief:
type: poststudy
file: surveys/debrief.json
annotation_schemes:
- annotation_type: likert
name: difficulty
description: The color-naming task was difficult.
size: 5
min_label: Strongly disagree
max_label: Strongly agree
label_requirement:
required: true
- annotation_type: likert
name: performance
description: I did well on the color-naming task.
size: 5
min_label: Strongly disagree
max_label: Strongly agree
label_requirement:
required: true
export_include_phase_data: true
```
`login.type: url_direct` logs each participant in with the `PROLIFIC_PID` from the study link, without a login page. `balance_on: completed` balances the two feedback conditions on participants who finished without being screened out. Participants who quit or fail the screener do not count toward either condition. From Potato 2.13.1, participants still working count toward their condition for `in_progress_minutes` after their last activity, 60 by default, so people who start at the same time are spread across both conditions. `save: incremental` stores each trial as the participant finishes it. With the default `save: on_finish`, closing the tab halfway through leaves no trials behind. `export_include_phase_data: true` puts the consent, screener, and debrief answers in the CSV export.
Every Potato study has an annotation phase with at least one item, and from 2.13.1 `potato validate` reports an error for a config without one. The Stroop study uses the phase for the two task ratings on a single item. `data/task.jsonl` has one line:
```json
{"id": "stroop_task", "text": "The color-naming task you just finished"}
```
## Consent and the color-vision screener
Prolific asks researchers to send participants who do not consent down a return path, and asks that such participants never be rejected ([Prolific's guide to screening within a study](https://researcher-help.prolific.com/en/articles/445165-can-i-screen-participants-within-my-study)). In Potato 2.13.1, every `exit_when` rule gives the same screen-out code. The consent page therefore has no `exit_when`. Participants who do not agree are asked to return the study on Prolific, and the screen-out code goes only to participants the screener excludes. `surveys/consent.json` reads:
```json
[
{
"name": "study_information",
"annotation_type": "pure_display",
"description": "About this study",
"labels": [
"You will press keys for the ink color of color words, then answer a few questions. The study takes about 10 minutes.",
"Your responses are stored on a university server under your Prolific ID.",
"If you do not want to take part, close this page and return the study on Prolific."
]
},
{
"name": "consent",
"annotation_type": "radio",
"description": "Do you agree to take part in this study?",
"labels": ["I agree to take part"],
"label_requirement": {"required_label": ["I agree to take part"]}
}
]
```
On a survey page, the server checks `required_label`, and from 2.13.1 the server checks `required: true` as well. In 2.12.1, `required: true` was checked only in the browser. A test request on 2.12.1 that skipped the consent question and posted the page directly got a 400 response and stayed on consent. The screener in `surveys/screener.json` lists both options under `required_label`. The server then requires an answer without requiring a particular one:
```json
[
{
"name": "color_vision",
"annotation_type": "radio",
"description": "Do you have difficulty telling red, green, or blue apart?",
"labels": ["Yes", "No"],
"label_requirement": {"required_label": ["Yes", "No"]}
}
]
```
A participant who answers "Yes" goes straight to the end of the session with `C1SCREEN`. The screener has its own page before the task. Potato checks an `exit_when` on a jsPsych page only after the timeline ends, when the participant has already done the task. Prolific also recommends filtering with its own prescreeners where possible, before participants reach the study.
## The Stroop timeline
The timeline file assigns a function to `window.potatoTimeline`. Potato creates the jsPsych instance, calls the function with the instance and a `potato` object, and saves the data when the timeline ends. The timeline below is adapted from the `examples/advanced/jspsych-stroop` example that ships with Potato:
```javascript
window.potatoTimeline = function (jsPsych, potato) {
var inks = [
{name: 'RED', color: '#c62828', key: 'r'},
{name: 'GREEN', color: '#2e7d32', key: 'g'},
{name: 'BLUE', color: '#1565c0', key: 'b'}
];
// Every word in every ink: 3 congruent and 6 incongruent pairs.
var stimuli = [];
inks.forEach(function (word) {
inks.forEach(function (ink) {
stimuli.push({
word: word.name,
ink: ink.name,
color: ink.color,
correct_response: ink.key,
congruent: word.name === ink.name
});
});
});
var instructions = {
type: jsPsychInstructions,
pages: [
'A color word will appear in colored ink.
' +
'Press the key for the ink color, not the word:
' +
'R = red, G = green, B = blue
',
'Answer as quickly and accurately as you can.
'
],
show_clickable_nav: true
};
var fixation = {
type: jsPsychHtmlKeyboardResponse,
stimulus: '+
',
choices: 'NO_KEYS',
trial_duration: function () {
return jsPsych.randomization.sampleWithoutReplacement([400, 500, 600], 1)[0];
},
data: {task: 'fixation'}
};
var trial = {
type: jsPsychHtmlKeyboardResponse,
stimulus: function () {
return '' +
jsPsych.evaluateTimelineVariable('word') + '
';
},
choices: ['r', 'g', 'b'],
trial_duration: 2500,
data: {
task: 'stroop',
word: jsPsych.timelineVariable('word'),
ink: jsPsych.timelineVariable('ink'),
congruent: jsPsych.timelineVariable('congruent'),
correct_response: jsPsych.timelineVariable('correct_response')
},
on_finish: function (data) {
data.correct = jsPsych.pluginAPI.compareKeys(data.response, data.correct_response);
}
};
// Runs only for participants in the feedback: immediate condition.
var feedback = {
timeline: [{
type: jsPsychHtmlKeyboardResponse,
stimulus: function () {
var last = jsPsych.data.get().last(1).values()[0];
if (last.response === null) { return 'Too slow
'; }
return last.correct ? 'Correct
' : 'Incorrect
';
},
choices: 'NO_KEYS',
trial_duration: 600,
data: {task: 'feedback'}
}],
conditional_function: function () {
return potato.conditions.feedback === 'immediate';
}
};
var block = {
timeline: [fixation, trial, feedback],
timeline_variables: stimuli,
repetitions: 2,
randomize_order: true
};
return [instructions, block];
};
```
`potato.conditions.feedback` is the level Potato assigned to the participant, and the `conditional_function` shows the feedback screen in the `immediate` condition only. The file does not call `initJsPsych` and sets no experiment-level `on_finish`, because Potato does both. Trial-level `on_finish` works as in any jsPsych study.
Potato 2.13.1 ships jsPsych 8.3.0 and 15 of its plugins inside the package, and the page loads both from the Potato server rather than from a CDN. The `plugins` list in the config names the two plugins the timeline uses. A plugin Potato does not ship, such as one from `jspsych-contrib`, goes in `extra_scripts` as a browser build. The 18 trials give each participant 6 congruent trials, and a real study raises `repetitions`.
## The ratings and the debrief
The two ratings run in the annotation phase, after the task. Both ratings set `label_requirement: required: true`. The server checks the requirement on annotation items, and a participant cannot reach the debrief without answering both. The debrief in `surveys/debrief.json` explains the manipulation and leaves the comment optional:
```json
[
{
"name": "debriefing",
"annotation_type": "pure_display",
"description": "What this study measured",
"labels": [
"Naming the ink color takes longer when the word names a different color. This slowdown is the Stroop effect. Half of the participants saw feedback after each response, and the study compares the Stroop effect between the two groups."
]
},
{
"name": "comments",
"annotation_type": "text",
"description": "Anything else you want to tell us? (optional)"
}
]
```
## Testing the study on your machine
Start the server from the project folder:
```bash
potato start config.yaml -p 8000
```
Open `http://localhost:8000/?PROLIFIC_PID=TEST1&STUDY_ID=s1&SESSION_ID=x1` and use a new `PROLIFIC_PID` for each test participant. Run through the study once in full, once answering "Yes" on the screener, and once reloading the page in the middle of the Stroop block. A participant who finishes sees `C1STROOP` with a "Return to Prolific" button, and a screened-out participant sees `C1SCREEN`. In 2.12.1, the screened-out participant's page shows the same "You have completed the annotation task" text as a finished participant's page, and the code is the only difference.
The jsPsych page has no Next button. Until the timeline finishes and the server has stored the run, the server answers any request to move on with an error, and a direct request to skip the page returned a 400 in testing. A reload starts the timeline over, because jsPsych cannot resume partway through. With `save: incremental`, the trials from before the reload stay in the export as a run with status `partial`, and the new run gets attempt number 2.
Test participants count toward the conditions and appear in the export. Delete `annotation_output/` and `project.sqlite` before launch.
## Putting the study on a server
Prolific participants reach the study only through a public address. [Production Setup](/docs/deployment/production-setup) covers running Potato behind nginx or Apache with TLS on a university server, and [Deploying Potato on a VPS](/docs/deployment/deploy-vps) covers a rented server. The trial data stays in `project.sqlite` in the project folder, and the ratings and survey answers stay in `annotation_output/`, both on the machine that runs the server.
On Prolific, choose to record Prolific IDs with URL parameters and set the study URL to your server's address with the three parameters:
```text
https://your-server.example/?PROLIFIC_PID={{%PROLIFIC_PID%}}&STUDY_ID={{%STUDY_ID%}}&SESSION_ID={{%SESSION_ID%}}
```
Add two completion paths on Prolific, `C1STROOP` for participants who finished and `C1SCREEN` with Prolific's "Screened out" label. Prolific pays screened-out participants the fixed screen-out reward you set, and their submissions do not count toward your places. Participants who do not consent return the study on Prolific and get no code. [Crowdsourcing Integration](/docs/deployment/crowdsourcing#prolific) has the Prolific API settings for tracking submissions.
## Exporting the trial data
Potato stores each jsPsych run in `project.sqlite` (table `runtime_runs`), not in the participants' `user_state.json` files. The `jspsych` export format writes the runs and trials as CSV:
```bash
python -m potato.export --config config.yaml --format jspsych --output export/
```
A test run on Potato 2.12.1 with seven simulated participants wrote these files:
| File | One row per | Contents |
|---|---|---|
| `jspsych_runs.csv` | run of the timeline | Participant, page, attempt, status (`complete` or `partial`), start and finish times, trial count, and `condition_feedback` |
| `jspsych_trials.csv` | jsPsych trial | `run_status`, `attempt`, `condition_feedback`, and every column jsPsych recorded, including `rt`, `response`, `plugin_version`, and the timeline's own `task`, `word`, `ink`, `congruent`, and `correct` |
| `jspsych_raw//stroop-.json` | run | The run's data as jsPsych produced it |
A fourth file, `jspsych_interaction.csv`, appears when jsPsych recorded blur, focus, or fullscreen events. The rows below are the first Stroop trials of one participant in the feedback condition, with a subset of the columns:
```text
user_id,attempt,run_status,trial_index,rt,response,condition_feedback,plugin_version,task,word,ink,congruent,correct
5f1a02,1,complete,2,604,r,immediate,2.2.0,stroop,BLUE,RED,False,True
5f1a02,1,complete,5,538,b,immediate,2.2.0,stroop,RED,RED,True,False
5f1a02,1,complete,8,545,g,immediate,2.2.0,stroop,BLUE,GREEN,False,True
```
`rt` is in milliseconds. On a trial that timed out, `rt` and `response` are empty and `correct` is `False`. jsPsych records the plugin's version on every trial, here 2.2.0 for `html-keyboard-response` and 2.1.0 for `instructions`. The jsPsych core version is not in the data, and the methods section should name both versions, such as "jsPsych 8.3.0, served by Potato 2.13.1". The CSV file has `True` and `False` for true and false values, and the raw JSON keeps JSON booleans.
The CSV export writes the rest of the study:
```bash
python -m potato.export --config config.yaml --format csv --output results/
```
`results/participants.csv` has one row per participant, with `condition_feedback`, `screened_out`, `screened_out_page`, and a `runtime_runs` column that gives the attempt count and status of the Stroop run. `results/annotations.csv` has the two ratings, with one column per scale point (`difficulty.1` to `difficulty.5`), and `results/phase_responses.csv` has the consent, screener, and debrief answers. For a data repository, `--format psych_ds` writes all five tables under `data/` with Psych-DS file names and a `dataset_description.json` that lists every column.
## Analyzing reaction times in R
The R code below reads the trials table, keeps the Stroop trials from complete runs, and computes each participant's Stroop effect as the mean correct reaction time on incongruent trials minus the mean on congruent trials:
```r
trials <- read.csv("export/jspsych_trials.csv")
stroop <- subset(trials, task == "stroop" & run_status == "complete")
stroop$congruent <- as.logical(stroop$congruent)
stroop$correct <- as.logical(stroop$correct)
accuracy <- aggregate(correct ~ user_id + condition_feedback, data = stroop, FUN = mean)
fast <- subset(stroop, correct & rt >= 200)
cells <- aggregate(rt ~ user_id + condition_feedback + congruent, data = fast, FUN = mean)
wide <- reshape(cells, idvar = c("user_id", "condition_feedback"),
timevar = "congruent", direction = "wide")
wide$stroop_effect <- wide$rt.FALSE - wide$rt.TRUE
aggregate(stroop_effect ~ condition_feedback, data = wide, FUN = mean)
t.test(stroop_effect ~ condition_feedback, data = wide)
```
`as.logical` reads the `True` and `False` strings and leaves a column `read.csv` has already converted unchanged. The 200 ms floor drops anticipatory key presses, and a preregistered cutoff replaces the floor where one exists. The accuracy table counts a timeout as an error, because a timed-out trial has `correct` set to false.
A participant whose complete run has `attempt` 2 reloaded the page partway through and saw some trials twice. `subset(stroop, attempt == 1)` keeps first attempts only, and the choice belongs in the analysis plan. For a trial-level model, [lme4](https://cran.r-project.org/package=lme4) fits the congruency effect, the feedback condition, and their interaction, with a random congruency slope for each participant:
```r
library(lme4)
fit <- lmer(rt ~ congruent * condition_feedback + (1 + congruent | user_id), data = fast)
summary(fit)
```
The `congruentTRUE:condition_feedbacknone` coefficient estimates how much the Stroop effect differs between the two feedback conditions. To count screen-outs for the participant flow, read `results/participants.csv` and tabulate `screened_out_page`.
## Where to go next
- [Coming from jsPsych](/docs/guides/coming-from-jspsych), for the concept map from jsPsych to Potato config keys
- [Runtime Phases](/docs/experiments/runtime-phases), for every key a `type: jspsych` page takes, including `assets_dir` for image and audio stimuli
- [Experiment Flow](/docs/experiments/experiment-flow), for conditions, `exit_when`, and counterbalanced page order
- [Online Behavioral Experiments](/docs/guides/online-behavioral-experiments), for timed trials on Potato's own items and SONA recruitment
- [Psychology experiments in Potato](/use-cases/psychology), for the behavioral features in one place
- [Behavioral Experiment Tools Compared](/docs/guides/behavioral-experiment-tools-compared), for jsPsych, PsychoPy, Gorilla, lab.js, and Qualtrics