Move a Qualtrics Survey to Potato Before Your License Ends

Survey researchers and lab managers whose university is ending or limiting Qualtrics access can follow this tutorial to move one survey from Qualtrics into Potato 2.13.1, from the .qsf export to a Prolific study and a CSV export. For example, Radboud University's Faculty of Social Sciences is phasing out Qualtrics and asks researchers to delete Qualtrics data and materials by December 1, 2026. The faculty points researchers to LimeSurvey. LimeSurvey is a good fit for a plain questionnaire, and Potato fits a study that varies what respondents see or asks respondents to rate items. The survey in this tutorial is an experiment of the second kind. Coming from Qualtrics maps each Qualtrics feature to a Potato key, and Survey Import lists the import flags. This page follows one survey through every step.

The example survey

The example is a headline credibility experiment written for this tutorial. The .qsf contains no real responses and no licensed material. Each respondent rates four news headlines, and a randomizer decides whether the headlines appear to come from a national newspaper or from a social media account. The survey flow runs in this order:

1. An embedded-data element stores PROLIFIC_PID from the study link.
2. A Consent block has skip logic to End of Survey for respondents who do not agree.
3. A Screener block asks about news reading and age, and a branch ends the survey for anyone who does not read news online.
4. A randomizer with Evenly Present Elements on presents one of two embedded-data elements and sets source to "a national newspaper" or "a social media account".
5. A Headlines block runs a Loop & Merge over the four headlines in random order. The question text pipes in ${e://Field/source} and ${lm://Field/1}, and a Timing question records time on the page.
6. A second randomizer shows a Media trust block and a News habits block in random order. Media trust is a matrix table, and a follow-up question in News habits has display logic on the "Social media" choice.
7. A Demographics block ends with an optional email question set to Request Response, a manipulation check, and a free-text question shown only when source is "a social media account".
8. A quota stops the survey at 400 completes, and both End of Survey elements redirect to Prolific with a completion code.

Exporting the .qsf and the responses from Qualtrics

A .qsf file contains the questions, blocks, flow, and logic of a survey. The file contains none of the responses, and Qualtrics exports the responses separately. Do both exports before your account closes:

1. In the survey's Survey tab, click Tools, choose Import/Export, and choose Export survey. Qualtrics downloads the .qsf file.
2. In the Data & Analysis tab, stay in the Data section, click Export & Import, and choose Export Data. Pick CSV, keep Download all fields selected, choose Export labels for the answer text or Export values for the numeric codes, and click Download.
3. If the survey collected file uploads or signatures, download the files from the same dialog under User Submitted Files.

If older analysis scripts read the numeric codes, export the responses a second time with values. Store every export where your institution's data policy says, before any deletion date. Potato stores each answer to a choice question as the option text. Export labels writes the same form, and old and new responses can then share one coding scheme in the analysis.

Importing the .qsf with potato import

Install the release this tutorial was checked against and run the import on the .qsf:

pip install potato-annotation==2.13.1
potato import -i headlines.qsf -o headline-study/

The importer detects the Qualtrics format from the file, and -f qualtrics names the format explicitly. The command prints a summary and a list of the files written:

Imported 7 page(s) and 15 question(s) from Qualtrics.
  8 construct(s) did not carry over; see IMPORT_REPORT.md:
    - quota QO_completes: quota: 400 completes
    - flow FL_2: embedded data: PROLIFIC_PID (recipient)
    - flow FL_6: end-of-survey redirect: https://app.prolific.com/submissions/complete?cc=C1SCREEN
    - block 'Headlines': loop and merge: looping by Static; the block is imported once
    - QID5 (accuracy): piped text: ${e://Field/source}, ${lm://Field/1} (shown as written)
    - QID7 (headline_timer): timing question: Potato records time on page and the browser itself
    - QID14 (email): request response: Qualtrics asks before moving on unanswered; Potato has no soft requirement
    - flow FL_15: end-of-survey redirect: https://app.prolific.com/submissions/complete?cc=C1HEADLN
  warning: the survey can screen participants out before its randomizer, but Potato assigns a condition on arrival, so conditions.balance_on is 'completed': screened-out and abandoned participants do not count toward a cell
  wrote headline-study/surveys/Consent.json
  wrote headline-study/surveys/Screener.json
  wrote headline-study/surveys/Headlines.json
  wrote headline-study/surveys/Media_trust.json
  wrote headline-study/surveys/News_habits.json
  wrote headline-study/surveys/Demographics.json
  wrote headline-study/surveys/Demographics_p2.json
  wrote headline-study/data/items.jsonl
  wrote headline-study/config.yaml
  wrote headline-study/IMPORT_REPORT.md
  wrote headline-study/README.md
 
Run it:
  potato start headline-study/config.yaml -p 8000

Each block became one page in surveys/, and the page break inside Demographics started a second page, Demographics_p2. config.yaml lists the pages in phases.order before a placeholder annotation task, with three placeholder items in data/items.jsonl. IMPORT_REPORT.md repeats the eight items and the warning above and adds a table that maps each Qualtrics question ID to a Potato question name.

Logic the import translated

Most of the survey's logic arrived as Potato rules. The consent skip logic and the screener branch became exit_when rules, the embedded-data randomizer became a between-subjects factor, and the second randomizer became a counterbalance group. An excerpt of the generated config.yaml shows all three:

conditions:
  factors:
    source:
    - a national newspaper
    - a social media account
  assignment: balanced
  balance_on: completed
phases:
  Screener:
    type: prestudy
    file: surveys/Screener.json
    exit_when:
      show_when:
      - schema: reads_news
        operator: equals
        value: 'No'
      logic: all
      message: Thank you. This is the end of the study.
  counterbalance:
  - pages:
    - Media_trust
    - News_habits
    method: random

assignment: balanced comes from Evenly Present Elements. The display logic that tested source now tests condition.source, and the "Social media" display logic became a display_logic rule on the platform question. The matrix table became a multirate question, Force Response became required: true, and the age question became a number question limited to 18 through 99. The "Somewhere else" text box became a free-response field, and choice randomization became randomize_order.

The imported study already runs. Start the study with potato start headline-study/config.yaml -p 8000 and read every page before editing anything.

Fixes for the reported items

Each of the eight report items needs a decision. The table pairs each item with the fix this tutorial makes, with the two redirects in one row:

Moving the looped block into the annotation phase

A Loop & Merge block repeats the same questions over a list of fields, and Potato's annotation phase repeats the annotation schemes over the items in a data file. The headlines therefore move out of the survey and into data/headlines.jsonl, one item per line:

{"id": "h1", "text": "Shared by {condition.source}: City council approves four-day school week starting in fall"}
{"id": "h2", "text": "Shared by {condition.source}: New study finds coffee drinkers live longer"}
{"id": "h3", "text": "Shared by {condition.source}: State to replace all lead water pipes by 2030"}
{"id": "h4", "text": "Shared by {condition.source}: Local hospital ends weekend emergency care"}

Potato replaces {condition.source} in the item text with the participant's level. A participant in the newspaper condition reads "Shared by a national newspaper: City council approves four-day school week starting in fall". The two questions from the Headlines block become the config's annotation_schemes, and surveys/Headlines.json and data/items.jsonl are deleted. assignment_strategy: random hands each participant all four headlines in a random order. The original survey randomized the loop order in the same way.

In Qualtrics, the trust, habits, and demographics blocks came after the headlines. Those pages now follow the annotation phase, and each page's type changes from prestudy to poststudy.

Consent and the screener

Prolific asks researchers to send participants who do not consent down a return path and never to reject them (Prolific's guide to screening within a study). In Potato 2.13.1, every exit_when rule gives the same screen-out code, and the consent refusal would receive the screener's code. The consent page therefore loses its exit_when rule and its "I do not agree" option, and the page text asks anyone who does not agree to return the study on Prolific.

The import also turned Force Response into required: true. From Potato 2.13.1, the server checks both required: true and required_label on a survey page, so a participant who posts the page some other way cannot skip either. In 2.12.1, required: true was checked in the browser only. Consent and the screener decide who reaches the study, so both questions use required_label. The consent question in surveys/Consent.json reads:

{
  "name": "consent",
  "annotation_type": "radio",
  "labels": ["I agree"],
  "description": "Do you agree to take part in this study?",
  "label_requirement": {"required_label": ["I agree"]}
}

The reads_news question in surveys/Screener.json lists both options, "required_label": ["Yes", "No"], and the server then requires an answer without requiring a particular one. The Screener's exit_when rule ends the study for anyone who answers "No".

Removing the email question

Prolific does not permit researchers to collect participants' personal email addresses without Prolific's prior approval. Prolific's guidance names updates about your research as one reason that does not qualify. Prolific asks researchers to contact participants through Prolific's messaging system. Deleting the email question from surveys/Demographics.json settles the Request Response item as well. Potato has no equivalent of Request Response, and a question without label_requirement is optional with no reminder.

Balancing conditions on completed participants

Qualtrics set source when a respondent reached the randomizer, after the screener. Potato assigns a condition when a participant first arrives, before the consent page. A participant screened out by the screener is therefore still assigned a cell, and under the default balance_on: assigned those screen-outs count toward the balance. The 2.13.1 importer sees the screener before the randomizer, sets balance_on: completed, and prints the warning shown in the import output. balance_on: completed counts participants who finished without being screened out, and participants still working within in_progress_minutes of their last activity, 60 by default. The 2.12.1 importer left balance_on at the default.

Smaller fixes

• Quota. Potato has no equivalent of a Qualtrics quota. The place count on the Prolific study caps the completes, and Prolific can raise the place count later.
• Timing. The headlines are annotation items now, and Potato records time per item in each participant's user_state.json. The Timing question is dropped.
• Slider wording. The 2.12.1 import appended the slider's statement label to the question text and produced "...watching news: Minutes per day", and the description in surveys/News_habits.json went back to the original question. The 2.13.1 import keeps the original question.

The finished config

The config below is complete and passes python -m potato.validate_cli config.yaml --strict on 2.12.1 and on 2.13.1:

annotation_task_name: Headline credibility study
task_dir: .
output_annotation_dir: annotation_output/
data_files:
  - data/headlines.jsonl
item_properties:
  id_key: id
  text_key: text
assignment_strategy: random
 
login:
  type: url_direct
  url_argument: PROLIFIC_PID
user_config:
  allow_all_users: true
  users: []
 
crowdsourcing:
  provider: prolific
  prolific:
    completion:
      code: C1HEADLN
      screened_out_code: C1SCREEN
 
conditions:
  factors:
    source: ["a national newspaper", "a social media account"]
  assignment: balanced
  balance_on: completed
 
phases:
  order: [Consent, Screener, annotation, Media_trust, News_habits, Demographics, Demographics_p2]
  Consent:
    type: prestudy
    file: surveys/Consent.json
  Screener:
    type: prestudy
    file: surveys/Screener.json
    exit_when:
      show_when:
        - {schema: reads_news, operator: equals, value: "No"}
  annotation:
    type: annotation
  Media_trust:
    type: poststudy
    file: surveys/Media_trust.json
  News_habits:
    type: poststudy
    file: surveys/News_habits.json
  Demographics:
    type: poststudy
    file: surveys/Demographics.json
  Demographics_p2:
    type: poststudy
    file: surveys/Demographics_p2.json
  counterbalance:
    - pages: [Media_trust, News_habits]
      method: random
 
annotation_schemes:
  - annotation_type: radio
    name: accuracy
    description: How accurate is the claim in this headline?
    labels: ["Not at all accurate", "Not very accurate", "Somewhat accurate", "Very accurate"]
    label_requirement:
      required: true
  - annotation_type: radio
    name: share
    description: Would you share this headline with a friend?
    labels: ["Yes", "No", "Not sure"]
    label_requirement:
      required: true
 
export_include_phase_data: true

The import set export_include_phase_data: true, and the setting stays. Without the setting, the export leaves out every answer given on a survey page, from consent to demographics, and writes no phase_responses.csv. The crowdsourcing block gives Prolific two codes, one for each redirect in the original survey. Participants who finish see C1HEADLN, and participants the screener's exit_when rule screens out see C1SCREEN. The import kept each rule's message, which Potato 2.12.1 does not display, and the finished config drops it.

Recruiting on Prolific or SONA

On Prolific, choose to record Prolific IDs through URL parameters and set the study URL to your server's address with the three parameters:

https://your-server.example/?PROLIFIC_PID={{%PROLIFIC_PID%}}&STUDY_ID={{%STUDY_ID%}}&SESSION_ID={{%SESSION_ID%}}

Add C1HEADLN and C1SCREEN to the Prolific study as completion codes, with Prolific's "Screened out" label on C1SCREEN. The done page sends each participant back to Prolific with one of the two. Participants who do not consent return the study on Prolific and get no code. Set the number of places to 400 to replace the quota.

For a SONA participant pool, replace the login and crowdsourcing blocks of the config above with these two blocks:

login:
  type: url_direct
  url_argument: sona_code
 
crowdsourcing:
  provider: sona
  sona:
    hostname: yourdept.sona-systems.com
    experiment_id: 123
    credit_token: your-credit-token

The hostname, experiment ID, and credit token come from the study information page in SONA. Set the SONA study URL to https://your-server.example/?sona_code=%SURVEY_CODE%. When a participant finishes, Potato grants the credit through SONA's server-side credit call, and the done page falls back to SONA's credit link if that call fails. Crowdsourcing Integration covers both providers in more detail.

Running and testing the study

Start the server from the project folder:

potato start config.yaml -p 8000

Before launch, walk through the study as a few test participants by opening http://localhost:8000/?PROLIFIC_PID=TEST1&STUDY_ID=s1&SESSION_ID=x1 with a new ID each time. Answer "No" on the screener once and check that the test participant sees the C1SCREEN code. A participant who finishes should see C1HEADLN, and the trust and habits pages should appear in both orders across participants. Test participants count toward the conditions and appear in the export. Stop the server and delete annotation_output/ before the real launch. Participants reach the study only through a public address. Deploying Potato on a VPS covers putting the server online.

Exporting the responses to CSV

Potato saves each participant's answers to annotation_output/<participant>/user_state.json as the study runs. The CSV export turns those files into three tables:

python -m potato.export --config config.yaml --format csv --output results/

The command writes results/annotations.csv, results/participants.csv, and results/phase_responses.csv:

A test run on Potato 2.12.1 with six simulated participants wrote these first rows of participants.csv and phase_responses.csv:

user_id,condition_source,condition_ordinal,cell_ordinal,latin_ordinal,final_phase,screened_out,screened_out_page,skipped_pages,page_orders,question_orders,runtime_runs,runtime_answers
5f00a9c,a social media account,0,0,,done,0,,,"{""group_0"": [""Media_trust"", ""News_habits""]}",{},{},{}
5f01a9c,a national newspaper,1,0,,done,0,,,"{""group_0"": [""News_habits"", ""Media_trust""]}",{},{},{}
 
user_id,phase,page,sequence,schema,label_name,value
5f00a9c,prestudy,Consent,0,consent,I agree,I agree
5f00a9c,prestudy,Screener,0,reads_news,Yes,Yes
5f00a9c,prestudy,Screener,1,age,age,20
5f00a9c,poststudy,Media_trust,0,trust,Newspapers,A great deal

R, Stata, and SPSS read all three files, and user_id joins the files. In annotations.csv, the accuracy rating for a row is the one accuracy. column that is filled. Rows in participants.csv with screened_out set to 1 have survey answers and no ratings.

Where to go next

• Coming from Qualtrics, for the Potato key behind each Qualtrics feature
• Survey Import, for the import flags and the six other survey formats
• Experiment Flow, for conditions, exit_when, and counterbalancing
• Crowdsourcing Integration, for the Prolific API and reclaiming abandoned work
• Survey research in Potato, for the survey features in one place
• Survey Research Tools Compared, for LimeSurvey, REDCap, and the other options