Conjoint Analysis: Immigrant Admission Preferences
Choice-based conjoint annotation modeled on the immigrant admission experiment in Hainmueller, Hopkins, and Yamamoto (Political Analysis, 2014), the paper that established the causal-inference framework for conjoint survey experiments. Respondents act as immigration officials: each task shows a pair of immigrant profiles with nine randomly assigned attributes (education, profession, country of origin, language skills, and more); the respondent makes a forced choice between the two profiles and then rates each on a 1-7 admission scale, exactly as in the paper's survey instrument.
About this dataset
Conjoint analysis presents respondents with multi-attribute profiles and asks them to choose or rate among them, allowing researchers to estimate the causal effect of each attribute on the overall choice. Hainmueller, Hopkins, and Yamamoto (2014) formalized this design in the potential-outcomes framework, introduced the average marginal component effect (AMCE), and showed it is nonparametrically identified under randomization assumptions that typical conjoint experiments satisfy.
The paper's running empirical examples are a presidential-candidate conjoint (311 respondents, six choice tasks each, eight candidate attributes) and an immigrant-admission conjoint fielded through Knowledge Networks, in which respondents acted as immigration officials, evaluated five pairings of immigrant profiles, made a forced choice within each pair, and rated each profile from 1 (should absolutely not admit) to 7 (should definitely admit).
The immigrant profiles combined nine attributes: gender, education level, language skills, country of origin, profession, job experience, employment plans, reason for application, and prior trips to the United States. Two randomization restrictions kept profiles plausible: applicants fleeing persecution came only from countries such as Iraq, Sudan, or Somalia, and high-skill professions required at least two years of college. Potato's conjoint scheme randomizes attributes independently, so this config does not enforce those restrictions.
This config uses Potato's conjoint scheme to generate the paired profiles from the paper's attribute-level sets (reproduced from Figure 3) and renders the forced-choice question below the profile cards, with two 1-7 likert items reproducing the per-profile rating questions. The annotation instructions are the survey instrument wording shown in Figure 1 of the paper. Because the design generates profiles from attributes, the sample-data items are self-authored task framings rather than dataset rows.
- Published
- Political Analysis 22(1):1-30, 2014
- Design
- Paired-profile choice-based conjoint with per-profile 1-7 ratings
- Immigrant attributes
- 9
- Pairings per respondent (immigrant experiment)
- 5
- Candidate experiment
- 311 respondents, 1,733 pairings, 8 attributes
Configuration Fileconfig.yaml
This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.
# Conjoint Analysis — Immigrant Admission Preferences (Paired Profiles)
# Based on the conjoint survey experiments in:
# Jens Hainmueller, Daniel J. Hopkins, and Teppei Yamamoto (2014), "Causal
# Inference in Conjoint Analysis: Understanding Multidimensional Choices via
# Stated Preference Experiments." Political Analysis 22(1):1-30.
# DOI: 10.1093/pan/mpt024
# Paper: https://doi.org/10.1093/pan/mpt024
#
# Task: respondents act as immigration officials. Each choice task presents a
# pair of immigrant profiles whose nine attributes (gender, education, language
# skills, country of origin, profession, job experience, job plans, reason for
# application, prior trips to the U.S.) are randomly assigned. The respondent
# makes a forced choice between the two profiles and then rates each profile on
# a 1-7 admission scale. Attribute names and levels reproduce Figure 3 of the
# paper. The paper also reports a companion presidential-candidate conjoint;
# this config implements the immigrant experiment, whose survey instrument the
# paper reproduces.
#
# Deliberate simplifications: the paper's randomization imposed two cross-
# attribute restrictions (profiles fleeing persecution came only from plausible
# countries such as Iraq, Sudan, or Somalia; high-skill professions required at
# least two years of college). Potato's conjoint scheme randomizes attributes
# independently, so these restrictions are not enforced. The two per-profile 1-7
# ratings are approximated with two likert schemes rendered outside the profile
# cards.
#
# Annotation instructions reproduced verbatim from Figure 1 ("Experimental
# Design: Immigration Conjoint") of the paper, which shows the survey instrument
# as fielded to Knowledge Networks respondents.
annotation_task_name: "Conjoint Analysis: Immigrant Admission Preferences"
task_dir: "."
data_files:
- sample-data.json
item_properties:
id_key: "id"
text_key: "text"
output_annotation_dir: "annotation_output/"
output_annotation_format: "json"
port: 8000
server_name: localhost
annotation_schemes:
- annotation_type: conjoint
name: immigrant_choice
description: "If you had to choose between them, which of these two immigrants should be given priority to come to the United States to live?"
profiles_per_set: 2
show_none_option: false
attributes:
- name: "Gender"
levels: ["female", "male"]
- name: "Education Level"
levels: ["no formal", "4th grade", "8th grade", "high school", "two-year college", "college degree", "graduate degree"]
- name: "Language Skills"
levels: ["fluent English", "broken English", "tried English but unable", "used interpreter"]
- name: "Country of Origin"
levels: ["Germany", "France", "Mexico", "Philippines", "Poland", "India", "China", "Sudan", "Somalia", "Iraq"]
- name: "Profession"
levels: ["janitor", "waiter", "child care provider", "gardener", "financial analyst", "construction worker", "teacher", "computer programmer", "nurse", "research scientist", "doctor"]
- name: "Job Experience"
levels: ["none", "1-2 years", "3-5 years", "5+ years"]
- name: "Employment Plans"
levels: ["contract with employer", "interviews with employer", "will look for work", "no plans to look for work"]
- name: "Reason for Application"
levels: ["reunite with family", "seek better job", "escape persecution"]
- name: "Prior Trips to the U.S."
levels: ["never", "once as tourist", "many times as tourist", "six months with family", "once w/o authorization"]
- annotation_type: likert
name: rating_immigrant_1
description: "On a scale from 1 to 7, where 1 indicates that the United States should absolutely not admit the immigrant and 7 indicates that the United States should definitely admit the immigrant, how would you rate Immigrant 1?"
min_label: "Absolutely Not Admit"
max_label: "Definitely Admit"
size: 7
- annotation_type: likert
name: rating_immigrant_2
description: "Using the same scale, how would you rate Immigrant 2?"
min_label: "Absolutely Not Admit"
max_label: "Definitely Admit"
size: 7
annotation_instructions: |
Please read the descriptions of the potential immigrants carefully. Then,
please indicate which of the two immigrants you would personally prefer to see
admitted to the United States.
[Two immigrant profiles are displayed side by side, each described by: Prior
Trips to the U.S., Reason for Application, Country of Origin, Language Skills,
Profession, Job Experience, Employment Plans, Education Level, and Gender.]
If you had to choose between them, which of these two immigrants should be
given priority to come to the United States to live?
On a scale from 1 to 7, where 1 indicates that the United States should
absolutely not admit the immigrant and 7 indicates that the United States
should definitely admit the immigrant, how would you rate Immigrant 1?
Using the same scale, how would you rate Immigrant 2?
html_layout: |
<div style="padding: 15px; max-width: 860px; margin: auto;">
<div style="background: #f0fdf4; border: 1px solid #bbf7d0; border-radius: 8px; padding: 12px 16px; margin-bottom: 10px;">
<p style="font-size: 15px; line-height: 1.6; margin: 0; color: #166534;">{{text}}</p>
</div>
<p style="font-size: 13px; color: #6b7280; margin: 0;">Please read the descriptions of the potential immigrants carefully before answering the questions below.</p>
</div>
allow_all_users: true
instances_per_annotator: 30
annotation_per_instance: 2
allow_skip: true
Sample Datasample-data.json
[
{
"id": "conjoint_001",
"text": "Choice task 1 of 10. This study asks you to act as an immigration official deciding which applicants should be admitted to the United States. Two immigrant profiles are shown below with randomly assigned characteristics."
},
{
"id": "conjoint_002",
"text": "Choice task 2 of 10. Please consider each applicant's full profile. There are no right or wrong answers; we are interested in your personal preference between the two applicants shown below."
}
]
// ... and 8 more itemsGet This Design
Clone or download from the repository
Quick start:
git clone https://github.com/davidjurgens/potato-showcase.git cd potato-showcase/text/computational-social-science/conjoint-candidate-profiles potato start config.yaml
Dataset & paper
Jens Hainmueller, Daniel J. Hopkins, and Teppei Yamamoto. 2014. Causal Inference in Conjoint Analysis: Understanding Multidimensional Choices via Stated Preference Experiments. Political Analysis 22(1):1-30.
Citation (BibTeX)
@article{hainmueller2014causal,
title = {Causal Inference in Conjoint Analysis: Understanding Multidimensional Choices via Stated Preference Experiments},
author = {Hainmueller, Jens and Hopkins, Daniel J. and Yamamoto, Teppei},
journal = {Political Analysis},
volume = {22},
number = {1},
pages = {1--30},
year = {2014},
publisher = {Cambridge University Press},
doi = {10.1093/pan/mpt024}
}Details
Annotation Types
Domain
Use Cases
Tags
Found an issue or want to improve this design?
Open an IssueRelated Designs
Politeness Transfer Annotation
Annotate text for politeness level, speech act type, and optional rewrite suggestions based on the Politeness Transfer framework. Annotators rate the politeness of workplace and email text on a 5-point scale and classify the communicative intent.
Warmth and Competence (Sentence-Level Social Perception)
Sentence-level annotation of social perception along the two fundamental dimensions of social cognition: warmth and competence. Following the W&C-Sent dataset (Ayesh, Mohammad, and Ousidhoum, ACL 2026), each item pairs a social-media sentence with a target entity (an individual or social group), and annotators rate how much the sentence expresses trust and sociability (the two components of warmth) and competence toward that target, each on a 7-point scale from -3 to +3. The dimensions are rated independently.
AlpacaFarm Preference Simulation
Simulate human preferences for instruction-following responses. Create preference data for efficient RLHF research and LLM evaluation.