Skip to content
Cette page n'est pas encore disponible dans votre langue. La version anglaise est affichée.

Multi-Phase Workflows

Surveys, training, और branching logic के साथ complex annotation workflows बनाएँ।

Multi-Phase Workflows

Potato 2.0 consent, pre-study surveys, instructions, training, annotation, और post-study feedback सहित कई sequential phases के साथ structured annotation workflows का समर्थन करता है।

उपलब्ध Phases

Phaseविवरण
consentInformed consent संग्रह
prestudyPre-annotation surveys (demographics, screening)
instructionsTask guidelines और जानकारी
trainingFeedback के साथ Practice questions
annotationMain annotation task (हमेशा आवश्यक)
poststudyPost-annotation surveys और feedback

बुनियादी कॉन्फ़िगरेशन

अपने कॉन्फ़िगरेशन में phases section का उपयोग करें:

yaml
phases:
  consent:
    enabled: true
    data_file: "data/consent.json"
 
  prestudy:
    enabled: true
    data_file: "data/demographics.json"
 
  instructions:
    enabled: true
    content: "data/instructions.html"
 
  training:
    enabled: true
    data_file: "data/training.json"
    schema_name: sentiment
    passing_criteria:
      min_correct: 8
 
  # annotation phase is always enabled
 
  poststudy:
    enabled: true
    data_file: "data/feedback.json"

Survey Question Types

Survey phases इन question types का समर्थन करते हैं:

Radio (Single Choice)

json
{
  "name": "experience",
  "type": "radio",
  "description": "How much annotation experience do you have?",
  "labels": ["None", "Some (< 10 hours)", "Moderate", "Extensive"],
  "required": true
}

Checkbox/Multiselect

json
{
  "name": "languages",
  "type": "checkbox",
  "description": "What languages do you speak fluently?",
  "labels": ["English", "Spanish", "French", "German", "Chinese", "Other"]
}

Text Input

json
{
  "name": "occupation",
  "type": "text",
  "description": "What is your occupation?",
  "required": true
}

Number Input

json
{
  "name": "years_experience",
  "type": "number",
  "description": "Years of professional experience",
  "min": 0,
  "max": 50
}

Likert Scale

json
{
  "name": "familiarity",
  "type": "likert",
  "description": "How familiar are you with this topic?",
  "size": 5,
  "min_label": "Not familiar",
  "max_label": "Very familiar"
}
json
{
  "name": "country",
  "type": "select",
  "description": "Select your country",
  "labels": ["USA", "Canada", "UK", "Germany", "France", "Other"]
}

शुरू करने से पहले informed consent एकत्र करें:

yaml
phases:
  consent:
    enabled: true
    data_file: "data/consent.json"

consent.json:

json
[
  {
    "name": "consent_agreement",
    "type": "radio",
    "description": "I have read and understood the research consent form and agree to participate.",
    "labels": ["I agree", "I do not agree"],
    "right_label": "I agree",
    "required": true
  }
]

right_label field आगे बढ़ने के लिए आवश्यक उत्तर निर्दिष्ट करता है।

Pre-Study Surveys

Demographics या screening questions एकत्र करें:

yaml
phases:
  prestudy:
    enabled: true
    data_file: "data/demographics.json"

demographics.json:

json
[
  {
    "name": "age_range",
    "type": "radio",
    "description": "What is your age range?",
    "labels": ["18-24", "25-34", "35-44", "45-54", "55+"],
    "required": true
  },
  {
    "name": "education",
    "type": "radio",
    "description": "Highest level of education completed",
    "labels": ["High school", "Bachelor's degree", "Master's degree", "Doctoral degree", "Other"],
    "required": true
  },
  {
    "name": "english_native",
    "type": "radio",
    "description": "Is English your native language?",
    "labels": ["Yes", "No"],
    "required": true
  }
]

Instructions Phase

Task instructions प्रदर्शित करें:

yaml
phases:
  instructions:
    enabled: true
    content: "data/instructions.html"

या inline content का उपयोग करें:

yaml
phases:
  instructions:
    enabled: true
    inline_content: |
      <h2>Task Instructions</h2>
      <p>In this task, you will classify the sentiment of product reviews.</p>
      <ul>
        <li><strong>Positive:</strong> Expresses satisfaction or praise</li>
        <li><strong>Negative:</strong> Expresses dissatisfaction or criticism</li>
        <li><strong>Neutral:</strong> Factual or mixed sentiment</li>
      </ul>

Training Phase

Feedback के साथ practice questions (विवरण के लिए Training Phase देखें):

yaml
phases:
  training:
    enabled: true
    data_file: "data/training.json"
    schema_name: sentiment
    passing_criteria:
      min_correct: 8
      total_questions: 10
    show_explanations: true

Post-Study Surveys

Annotation के बाद feedback एकत्र करें:

yaml
phases:
  poststudy:
    enabled: true
    data_file: "data/feedback.json"

feedback.json:

json
[
  {
    "name": "difficulty",
    "type": "likert",
    "description": "How difficult was this task?",
    "size": 5,
    "min_label": "Very easy",
    "max_label": "Very difficult"
  },
  {
    "name": "clarity",
    "type": "likert",
    "description": "How clear were the instructions?",
    "size": 5,
    "min_label": "Very unclear",
    "max_label": "Very clear"
  },
  {
    "name": "suggestions",
    "type": "text",
    "description": "Any suggestions for improvement?",
    "textarea": true,
    "required": false
  }
]

Built-in Templates

Potato में सामान्य survey questions के लिए predefined label sets शामिल हैं:

TemplateLabels
countriesदेशों की सूची
languagesसामान्य भाषाएँ
ethnicityEthnicity options
religionReligion options

अपने questions में templates का उपयोग करें:

json
{
  "name": "country",
  "type": "select",
  "description": "Select your country",
  "template": "countries"
}

Free Response Fields

Structured questions के साथ optional text input जोड़ें:

json
{
  "name": "topics",
  "type": "checkbox",
  "description": "Which topics interest you?",
  "labels": ["Technology", "Sports", "Politics", "Entertainment"],
  "free_response": true,
  "free_response_label": "Other (please specify)"
}

Page Headers

Survey section headers customize करें:

json
{
  "page_header": "Demographics Survey",
  "questions": [
    {"name": "age", "type": "radio", ...},
    {"name": "gender", "type": "radio", ...}
  ]
}

पूर्ण उदाहरण

yaml
task_name: "Sentiment Analysis Study"
task_dir: "."
port: 8000
 
# Data configuration
data_files:
  - "data/reviews.json"
 
item_properties:
  id_key: id
  text_key: text
 
# Annotation scheme
annotation_schemes:
  - annotation_type: radio
    name: sentiment
    description: "What is the sentiment of this review?"
    labels:
      - Positive
      - Negative
      - Neutral
    sequential_key_binding: true
 
# Multi-phase workflow
phases:
  consent:
    enabled: true
    data_file: "data/consent.json"
 
  prestudy:
    enabled: true
    data_file: "data/demographics.json"
 
  instructions:
    enabled: true
    content: "data/instructions.html"
 
  training:
    enabled: true
    data_file: "data/training.json"
    schema_name: sentiment
    passing_criteria:
      min_correct: 8
      total_questions: 10
    retries:
      enabled: true
      max_retries: 2
    show_explanations: true
 
  # annotation phase is always enabled
 
  poststudy:
    enabled: true
    data_file: "data/feedback.json"
 
# Output
output_annotation_dir: "output/"
output_annotation_format: "json"
 
# User access
allow_all_users: true

Legacy Configuration

पुराना surveyflow configuration format backward compatibility के लिए अभी भी समर्थित है:

yaml
surveyflow:
  enabled: true
  phases:
    - name: pre_survey
      type: survey
      questions: survey_questions.json
    - name: main_annotation
      type: annotation

हालाँकि, हम नए projects के लिए नए phases format पर migrate करने की सिफारिश करते हैं।

सर्वोत्तम प्रथाएँ

1. Surveys को संक्षिप्त रखें

लंबे surveys completion rates कम करते हैं। केवल आवश्यक questions पर ध्यान केंद्रित करें।

2. जटिल Tasks के लिए Training का उपयोग करें

Training phases annotation quality सुधारते हैं, विशेष रूप से nuanced tasks के लिए।

3. उचित Passing Criteria निर्धारित करें

yaml
# Too strict - may exclude good annotators
passing_criteria:
  require_all_correct: true
 
# Better - allows for learning
passing_criteria:
  min_correct: 8
  total_questions: 10

4. स्पष्ट Instructions प्रदान करें

Expectations clarify करने के लिए अपने instructions phase में examples शामिल करें।

5. पूरे Flow को Test करें

Issues पकड़ने के लिए deployment से पहले पूरे workflow को स्वयं पूरा करें।

6. Required Fields का बुद्धिमानी से उपयोग करें

केवल उन्हीं questions को required mark करें जो essential हैं - optional questions बेहतर response quality प्राप्त करते हैं।

Crowdsourcing Integration

Prolific या MTurk के लिए, completion codes configure करें:

yaml
phases:
  poststudy:
    enabled: true
    data_file: "data/feedback.json"
    show_completion_code: true
    completion_code_format: "POTATO-{user_id}-{timestamp}"

अधिक विवरण के लिए Crowdsourcing देखें।