Multi-Phase Workflows
Surveys, training, और branching logic के साथ complex annotation workflows बनाएँ।
Multi-Phase Workflows
Potato 2.0 consent, pre-study surveys, instructions, training, annotation, और post-study feedback सहित कई sequential phases के साथ structured annotation workflows का समर्थन करता है।
उपलब्ध Phases
| Phase | विवरण |
|---|---|
consent | Informed consent संग्रह |
prestudy | Pre-annotation surveys (demographics, screening) |
instructions | Task guidelines और जानकारी |
training | Feedback के साथ Practice questions |
annotation | Main annotation task (हमेशा आवश्यक) |
poststudy | Post-annotation surveys और feedback |
बुनियादी कॉन्फ़िगरेशन
अपने कॉन्फ़िगरेशन में phases section का उपयोग करें:
phases:
consent:
enabled: true
data_file: "data/consent.json"
prestudy:
enabled: true
data_file: "data/demographics.json"
instructions:
enabled: true
content: "data/instructions.html"
training:
enabled: true
data_file: "data/training.json"
schema_name: sentiment
passing_criteria:
min_correct: 8
# annotation phase is always enabled
poststudy:
enabled: true
data_file: "data/feedback.json"Survey Question Types
Survey phases इन question types का समर्थन करते हैं:
Radio (Single Choice)
{
"name": "experience",
"type": "radio",
"description": "How much annotation experience do you have?",
"labels": ["None", "Some (< 10 hours)", "Moderate", "Extensive"],
"required": true
}Checkbox/Multiselect
{
"name": "languages",
"type": "checkbox",
"description": "What languages do you speak fluently?",
"labels": ["English", "Spanish", "French", "German", "Chinese", "Other"]
}Text Input
{
"name": "occupation",
"type": "text",
"description": "What is your occupation?",
"required": true
}Number Input
{
"name": "years_experience",
"type": "number",
"description": "Years of professional experience",
"min": 0,
"max": 50
}Likert Scale
{
"name": "familiarity",
"type": "likert",
"description": "How familiar are you with this topic?",
"size": 5,
"min_label": "Not familiar",
"max_label": "Very familiar"
}Dropdown Select
{
"name": "country",
"type": "select",
"description": "Select your country",
"labels": ["USA", "Canada", "UK", "Germany", "France", "Other"]
}Consent Phase
शुरू करने से पहले informed consent एकत्र करें:
phases:
consent:
enabled: true
data_file: "data/consent.json"consent.json:
[
{
"name": "consent_agreement",
"type": "radio",
"description": "I have read and understood the research consent form and agree to participate.",
"labels": ["I agree", "I do not agree"],
"right_label": "I agree",
"required": true
}
]right_label field आगे बढ़ने के लिए आवश्यक उत्तर निर्दिष्ट करता है।
Pre-Study Surveys
Demographics या screening questions एकत्र करें:
phases:
prestudy:
enabled: true
data_file: "data/demographics.json"demographics.json:
[
{
"name": "age_range",
"type": "radio",
"description": "What is your age range?",
"labels": ["18-24", "25-34", "35-44", "45-54", "55+"],
"required": true
},
{
"name": "education",
"type": "radio",
"description": "Highest level of education completed",
"labels": ["High school", "Bachelor's degree", "Master's degree", "Doctoral degree", "Other"],
"required": true
},
{
"name": "english_native",
"type": "radio",
"description": "Is English your native language?",
"labels": ["Yes", "No"],
"required": true
}
]Instructions Phase
Task instructions प्रदर्शित करें:
phases:
instructions:
enabled: true
content: "data/instructions.html"या inline content का उपयोग करें:
phases:
instructions:
enabled: true
inline_content: |
<h2>Task Instructions</h2>
<p>In this task, you will classify the sentiment of product reviews.</p>
<ul>
<li><strong>Positive:</strong> Expresses satisfaction or praise</li>
<li><strong>Negative:</strong> Expresses dissatisfaction or criticism</li>
<li><strong>Neutral:</strong> Factual or mixed sentiment</li>
</ul>Training Phase
Feedback के साथ practice questions (विवरण के लिए Training Phase देखें):
phases:
training:
enabled: true
data_file: "data/training.json"
schema_name: sentiment
passing_criteria:
min_correct: 8
total_questions: 10
show_explanations: truePost-Study Surveys
Annotation के बाद feedback एकत्र करें:
phases:
poststudy:
enabled: true
data_file: "data/feedback.json"feedback.json:
[
{
"name": "difficulty",
"type": "likert",
"description": "How difficult was this task?",
"size": 5,
"min_label": "Very easy",
"max_label": "Very difficult"
},
{
"name": "clarity",
"type": "likert",
"description": "How clear were the instructions?",
"size": 5,
"min_label": "Very unclear",
"max_label": "Very clear"
},
{
"name": "suggestions",
"type": "text",
"description": "Any suggestions for improvement?",
"textarea": true,
"required": false
}
]Built-in Templates
Potato में सामान्य survey questions के लिए predefined label sets शामिल हैं:
| Template | Labels |
|---|---|
countries | देशों की सूची |
languages | सामान्य भाषाएँ |
ethnicity | Ethnicity options |
religion | Religion options |
अपने questions में templates का उपयोग करें:
{
"name": "country",
"type": "select",
"description": "Select your country",
"template": "countries"
}Free Response Fields
Structured questions के साथ optional text input जोड़ें:
{
"name": "topics",
"type": "checkbox",
"description": "Which topics interest you?",
"labels": ["Technology", "Sports", "Politics", "Entertainment"],
"free_response": true,
"free_response_label": "Other (please specify)"
}Page Headers
Survey section headers customize करें:
{
"page_header": "Demographics Survey",
"questions": [
{"name": "age", "type": "radio", ...},
{"name": "gender", "type": "radio", ...}
]
}पूर्ण उदाहरण
task_name: "Sentiment Analysis Study"
task_dir: "."
port: 8000
# Data configuration
data_files:
- "data/reviews.json"
item_properties:
id_key: id
text_key: text
# Annotation scheme
annotation_schemes:
- annotation_type: radio
name: sentiment
description: "What is the sentiment of this review?"
labels:
- Positive
- Negative
- Neutral
sequential_key_binding: true
# Multi-phase workflow
phases:
consent:
enabled: true
data_file: "data/consent.json"
prestudy:
enabled: true
data_file: "data/demographics.json"
instructions:
enabled: true
content: "data/instructions.html"
training:
enabled: true
data_file: "data/training.json"
schema_name: sentiment
passing_criteria:
min_correct: 8
total_questions: 10
retries:
enabled: true
max_retries: 2
show_explanations: true
# annotation phase is always enabled
poststudy:
enabled: true
data_file: "data/feedback.json"
# Output
output_annotation_dir: "output/"
output_annotation_format: "json"
# User access
allow_all_users: trueLegacy Configuration
पुराना surveyflow configuration format backward compatibility के लिए अभी भी समर्थित है:
surveyflow:
enabled: true
phases:
- name: pre_survey
type: survey
questions: survey_questions.json
- name: main_annotation
type: annotationहालाँकि, हम नए projects के लिए नए phases format पर migrate करने की सिफारिश करते हैं।
सर्वोत्तम प्रथाएँ
1. Surveys को संक्षिप्त रखें
लंबे surveys completion rates कम करते हैं। केवल आवश्यक questions पर ध्यान केंद्रित करें।
2. जटिल Tasks के लिए Training का उपयोग करें
Training phases annotation quality सुधारते हैं, विशेष रूप से nuanced tasks के लिए।
3. उचित Passing Criteria निर्धारित करें
# Too strict - may exclude good annotators
passing_criteria:
require_all_correct: true
# Better - allows for learning
passing_criteria:
min_correct: 8
total_questions: 104. स्पष्ट Instructions प्रदान करें
Expectations clarify करने के लिए अपने instructions phase में examples शामिल करें।
5. पूरे Flow को Test करें
Issues पकड़ने के लिए deployment से पहले पूरे workflow को स्वयं पूरा करें।
6. Required Fields का बुद्धिमानी से उपयोग करें
केवल उन्हीं questions को required mark करें जो essential हैं - optional questions बेहतर response quality प्राप्त करते हैं।
Crowdsourcing Integration
Prolific या MTurk के लिए, completion codes configure करें:
phases:
poststudy:
enabled: true
data_file: "data/feedback.json"
show_completion_code: true
completion_code_format: "POTATO-{user_id}-{timestamp}"अधिक विवरण के लिए Crowdsourcing देखें।