Survey Research Tools Compared

LimeSurvey is a free, open-source replacement for a Qualtrics questionnaire, with quotas and exports to SPSS, R, and Stata, while REDCap suits clinical research databases, KoboToolbox suits offline field interviews, SurveyJS suits forms inside your own application, and oTree suits interactive economic experiments. Potato is a free, self-hosted alternative to Qualtrics for survey experiments. Potato imports Qualtrics, LimeSurvey, and REDCap questionnaires and adds balanced random assignment, factorial vignettes, list experiments, and Prolific or SONA recruitment.

A survey experiment gives each respondent one version of a questionnaire by random assignment and compares answers across the versions. A vignette is a short scenario whose details change from one condition to another, and a factorial vignette design crosses several such details. Conjoint analysis asks respondents to choose between profiles built from attribute levels and estimates how much each attribute changes the choice. A list experiment and randomized response let respondents answer a sensitive question without revealing the respondent's own answer. Data residency is the country and the server where the responses are stored.

Many researchers start this comparison when a university limits Qualtrics. Radboud University's Faculty of Social Sciences chose not to renew the faculty's Qualtrics license, first for cost and then on the advice of the university's Data Protection Officer. The faculty asks researchers to move to LimeSurvey by December 1, 2026, and to delete Qualtrics data and materials by the same date. Survey research in Potato covers Potato's survey features in one place, and Behavioral Experiment Tools Compared covers software for reaction-time tasks.

Which survey tools are worth comparing?

Costs come from each vendor's own pricing page in October 2026, and papers describing REDCap (2009), oTree (2016), and Potato 2.0 (2026) are listed under References.

Which tool for which survey

The table pairs each kind of study with the tools built for that kind of study.

Qualtrics appears in many rows because one hosted product covers randomization and quotas as well as questionnaires. Potato appears where a study varies what respondents see, asks sensitive questions, or puts items in front of respondents to rate or label.

Hosting, data residency, and cost

Qualtrics hosts every survey on its own servers. LimeSurvey's Community Edition stores responses on your own server or your provider's, and LimeSurvey Cloud hosts accounts in Germany, Finland, the United Kingdom, the United States, Canada, and Australia. LimeSurvey recommends Germany or Finland for EU customers. REDCap runs on your institution's servers. A member institution signs an end-user license agreement with Vanderbilt University and supports REDCap through its own IT department. KoboToolbox hosts accounts on a Global server or on a European Union server in Ireland.

SurveyJS sends responses to a backend you build, and its pricing page states that all data stays on your own servers. oTree's documentation describes deployment to Heroku through oTree Hub and to Ubuntu or Windows servers you manage. Potato runs on your own machine or your institution's server and writes each participant's answers to user_state.json in the study's output directory. Potato contacts an outside service only when the config names the service, as with Prolific's API or a hosted AI endpoint.

Free Qualtrics alternatives

LimeSurvey's Community Edition, REDCap for consortium members, the SurveyJS Form Library, oTree, and Potato cost nothing to license. Running any of the five means paying for a server and for the staff time to maintain the server. Qualtrics, LimeSurvey Cloud, and KoboToolbox's paid plans charge for the hosting instead. Each free tool covers part of what a Qualtrics account does:

• LimeSurvey Community Edition comes closest for an ordinary questionnaire, with quotas, several languages in one survey, and exports to SPSS, R, and Stata. Random assignment takes rand() in an equation question, and we found no conjoint module in LimeSurvey's documentation.
• REDCap is free only to non-profit members of the REDCap consortium. REDCap suits clinical and longitudinal research and randomizes from an allocation table you upload.
• The SurveyJS Form Library is free under the MIT license, and the Survey Creator costs from $589 per developer. Responses go to a backend you build.
• KoboToolbox has a free Community plan with usage limits and collects interviews offline. We found no random assignment to conditions in KoboToolbox's documentation.
• oTree is free and runs interactive experiments with groups and payoffs, and every assignment rule is Python code you write.
• Potato imports a Qualtrics .qsf and runs balanced assignment, vignettes, conjoint, and list experiments. Potato has no quotas, no multi-language surveys, and no SPSS files.

Survey experiments, recruitment, and exports by tool

The second table compares the parts of a study around the questions. SurveyJS is left out because every row depends on the application you build around the library.

A dash means we found nothing for the row in the tool's own documentation, checked on 2026-10-28. Prolific states that most survey software works with Prolific when the study link records the Prolific ID and sends the participant back at the end. A dash in the Prolific row therefore means no published guide. A list experiment can be built in any tool with random assignment and display logic, with the estimate computed afterward. If a cell is wrong, tell us.

Potato's sensitive-question designs and assignment are covered in Sensitive Questions and Experiment Flow. The survey_experiments export writes a difference-in-means estimate for each list experiment and a prevalence estimate for each randomized-response question, with a standard error and a 95% interval for every estimate. participants.csv writes each participant's condition, page order, question order, and any screen-out. The analysis file therefore records what each respondent was assigned and shown.

Moving a questionnaire into Potato

potato import turns the questions, pages, and show/hide logic of a questionnaire into Potato survey pages and rules. The command reads seven formats:

• a Qualtrics .qsf file;
• a LimeSurvey .lss file;
• a REDCap data dictionary;
• SurveyJS JSON;
• an XLSForm, as used by ODK, KoboToolbox, SurveyCTO, and Ona;
• a Beiwe survey; and
• a Network Canvas protocol.

potato import -i my-survey.qsf -o survey-project/

A .qsf file contains a survey's questions, blocks, flow, and logic, and Qualtrics exports the file from Survey → Tools → Import/Export → Export Survey. None of the seven formats contains responses. Export the response data separately before an account closes. Everything the importer could not translate, such as Qualtrics quotas, piped text, or REDCap calculated fields, is listed in IMPORT_REPORT.md. Survey Import lists what imports from each format, and Coming from Qualtrics maps Qualtrics blocks, randomizers, and End of Survey elements to Potato keys.

A factorial vignette experiment in Potato

The config below runs a 2×2 vignette experiment recruited on Prolific. Each respondent is assigned one combination of sponsor and funding source on arrival, and {condition.sponsor} and {condition.funding} in the item text are filled with the respondent's levels. balance_on: completed counts respondents who finish and respondents still working within in_progress_minutes of their last activity, so dropouts and screen-outs do not unbalance the four cells. The screener ends the study for anyone outside the United States and returns the respondent to Prolific with the screen-out code. Two published instruments follow the vignettes, with the efficacy items in a different order for each respondent.

annotation_task_name: Local policy vignette experiment
task_dir: .
output_annotation_dir: annotation_output/
data_files:
  - data/vignettes.jsonl
item_properties:
  id_key: id
  text_key: text
 
login:
  type: url_direct
  url_argument: PROLIFIC_PID
 
conditions:
  factors:
    sponsor: ["The city council", "A neighborhood association"]
    funding: ["existing city funds", "a new property tax"]
  assignment: balanced
  balance_on: completed
 
phases:
  order: [consent, screener, annotation, efficacy, demographics]
  consent:
    type: consent
    file: surveys/consent.json
  screener:
    type: prestudy
    file: surveys/screener.json
    exit_when:
      show_when:
        - {schema: us_resident, operator: equals, value: "No"}
      message: This study is for people who live in the United States. Thank you for your time.
  efficacy:
    type: poststudy
    instrument: political-efficacy
    randomize_questions: true
  demographics:
    type: poststudy
    instrument: anes-demographics
 
crowdsourcing:
  provider: prolific
  prolific:
    completion:
      code: C1VIGNETTE
      screened_out_code: C1SCREENED
 
annotation_schemes:
  - annotation_type: likert
    name: support
    description: How strongly do you support or oppose this proposal?
    size: 7
    min_label: Strongly oppose
    max_label: Strongly support
    label_requirement:
      required: true
  - annotation_type: radio
    name: sponsor_check
    description: Who made this proposal?
    labels: ["The city council", "A neighborhood association", "I don't remember"]
    label_requirement:
      required: true
 
# Exports leave out survey answers unless export_include_phase_data is true
export_include_phase_data: true

Each line of data/vignettes.jsonl is one vignette, with the placeholders in its text:

{"id": "v1", "text": "{condition.sponsor} proposes turning the parking lot on Mill Street into a public park, paid for by {condition.funding}."}

The screener page in surveys/screener.json asks one question. With required_label, the server keeps the respondent on the page until the respondent picks an answer, and the screen-out rule always has an answer to test:

[
  {"name": "us_resident", "annotation_type": "radio",
   "description": "Do you currently live in the United States?",
   "labels": ["Yes", "No"],
   "label_requirement": {"required_label": ["Yes", "No"]}}
]

consent.json follows the same format, with required_label: ["I agree"] on the consent question. The sponsor question after each vignette is a manipulation check. Run python -m potato.export --config config.yaml --format csv --output results/ for annotations.csv with the ratings, participants.csv with each respondent's cell and screen-out, and phase_responses.csv with the consent, screener, and instrument answers.

Coding open-ended answers

Open-ended answers are where many survey projects bring in qualitative data analysis software. NVivo and ATLAS.ti are sold under paid licenses with student pricing. The tools below differ in how each reads a survey export. Qualitative Data Analysis Software Compared compares the same tools as coding software.

• NVivo imports a Qualtrics survey directly. Each respondent becomes a case, closed questions become case attributes, and the answers to each open-ended question are grouped into one code. A survey exported as CSV can be imported as a spreadsheet instead, with fewer features.
• ATLAS.ti imports a survey from an Excel, CSV, or ODS file. Each open-ended question becomes a code, and each answer becomes a quotation coded with the question, the respondent, and the respondent's closed answers.
• Usercall runs AI-moderated interviews and turns transcripts and imported open-ended answers into themes with supporting excerpts. Plans start at $89 a month.
• Potato loads exported answers as items in a data file, splits the items among several coders working from a codebook, and reports Krippendorff's alpha per question with a bootstrap interval. Content Analysis walks through the setup.

What Potato does not do for survey research

• No panel of its own. Potato recruits through Prolific, CloudResearch Connect, SONA, and other panels that pass an ID in the URL. A study that needs a probability sample or a national panel buys access from a panel provider.
• No quotas on respondent answers. max_per_cell caps how many respondents each condition receives, and Potato has no cap on respondents by age, gender, or region. Qualtrics and LimeSurvey have quotas.
• No offline survey pages. Consent, screener, and survey pages need a connection to the server. Pocket Mode lets annotators label prefetched items offline on a phone, but the mode applies to the annotation phase only. KoboToolbox and the REDCap mobile app collect whole interviews offline.
• No field data types. GPS points, photos, audio recordings, and barcodes from an XLSForm are reported by the import and not brought over.
• No payments of its own. Potato computes bonuses from price lists, BDM bids, and belief elicitation. Prolific pays the bonuses, from an uploaded bulk-bonus file or a request sent from Potato's admin route. Incentive Payments covers the rules.
• One language per study. ui_language translates Potato's interface into one of ten bundled languages besides English, and the questions are written in that language. A survey offered in three languages is three configs.
• No questionnaire without items. Every Potato study has an annotation phase over items from a data file, and from version 2.13.1 potato validate reports an error for a config without one. A plain questionnaire uses one item with a single-label radio scheme such as Finish, with the annotation phase placed last. A pure_display item does not work for the purpose, because it never completes.
• No SPSS, SAS, or Stata files. Exports are CSV, TSV, JSONL, Parquet, a Frictionless Data Package, or Psych-DS. SPSS, SAS, Stata, and R all read the CSV files.
• Scheduling fixed at the first visit. From version 2.13, a schedule: block runs follow-up waves, experience-sampling prompts, and Delphi rounds, and a notifications: block sends reminders by email, SMS, or Prolific message. Each participant's schedule is computed when the first visit ends and does not change afterwards. REDCap schedules repeated instruments and events.

Checked against each project's own documentation in October 2026.

Frequently asked questions

Is there a free, self-hosted alternative to Qualtrics?

Yes. LimeSurvey's Community Edition and Potato are free, open-source survey servers you run yourself, and REDCap is free and installed by your institution if the institution belongs to the REDCap consortium. LimeSurvey is the tool Radboud's Faculty of Social Sciences chose for general questionnaires, with quotas and SPSS, R, and Stata exports. Potato imports a Qualtrics .qsf and adds balanced assignment, vignettes, list experiments, and Prolific or SONA recruitment, without quotas or SPSS files.

Should I use REDCap or Qualtrics for a research survey?

Use REDCap for clinical, health, or longitudinal studies at an institution that already runs REDCap. REDCap keeps an audit trail, schedules repeated instruments and events, collects data offline in its mobile app, and costs consortium members nothing. Qualtrics suits general questionnaires and survey experiments, with quotas, a Randomizer for balanced conditions, specialty question types such as heat maps, and conjoint projects as an additional purchase. Potato imports questionnaires from both.

What should I do before my university's Qualtrics license ends?

Export the survey and the responses separately before the account closes. Qualtrics writes the questionnaire to a .qsf file that contains the questions, logic, and flow but no responses, and the response data comes out of the Data & Analysis tab as CSV, SPSS, or another format. The .qsf can be imported into Potato with potato import, and the response files go to your institution's archive or data repository. Finish both exports before any deletion date your university sets, such as December 1, 2026, at Radboud's Faculty of Social Sciences.

Further reading

• Survey research in Potato, for the survey features in one place
• Survey Import and Coming from Qualtrics
• Move a Qualtrics Survey to Potato Before Your License Ends, for one survey moved from the .qsf export to a Prolific study and a CSV export
• Experiment Flow, for conditions, page rules, screen-outs, and counterbalancing
• Sensitive Questions, for list experiments and randomized response
• Survey Instruments
• Running a Crowdsourced Study
• Export Formats
• Behavioral Experiment Tools Compared
• Annotation Tools Compared

References

1. Harris, P. A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N., & Conde, J. G. (2009). Research electronic data capture (REDCap): A metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, 42(2), 377-381. https://doi.org/10.1016/j.jbi.2008.08.010
2. Chen, D. L., Schonger, M., & Wickens, C. (2016). oTree: An open-source platform for laboratory, online, and field experiments. Journal of Behavioral and Experimental Finance, 9, 88-97. https://doi.org/10.1016/j.jbef.2015.12.001
3. Jurgens, D., Chen, M., & Iyer, L. (2026). Potato 2.0: A Comprehensive Annotation Platform with AI-in-the-Loop Support. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), 374-386. https://aclanthology.org/2026.acl-demo.37/