Docs/Core Concepts

User Management

Configure user access, authentication, and collaboration settings.

User Management

Potato provides flexible user management options, from open access to restricted authentication.

Access Modes

Open Access

Allow anyone to annotate without login:

allow_all_users: true

Restricted Access

Limit to specific users:

allow_all_users: false
authorized_users:
  - alice@example.com
  - bob@example.com
  - researcher@university.edu

Authentication

Email-Based Login

Users enter their email to access the annotation interface:

allow_all_users: false
authorized_users:
  - user1@example.com
  - user2@example.com

Crowdsourcing Integration

For Prolific or MTurk workers:

# Prolific integration
allow_all_users: true
prolific_integration: true
prolific_completion_code: "ABC123"
 
# Workers authenticated via URL parameter
# https://yourserver.com/?PROLIFIC_PID=xxx

URL Parameter Authentication

Pass user ID via URL:

url_user_id_param: user_id
# Access via: http://localhost:8000/?user_id=annotator1

User Roles

Annotators

Regular users who label data:

authorized_users:
  - annotator1@example.com
  - annotator2@example.com

Administrators

Users with access to admin dashboard:

admin_users:
  - admin@example.com
  - lead_researcher@university.edu

Admins can:

  • View all annotations
  • Monitor progress
  • Export data
  • Manage users

Task Assignment

Instances Per User

Limit how many items each user annotates:

instances_per_annotator: 100

Annotations Per Instance

Require multiple annotators per item:

annotation_per_instance: 3

Overlap Settings

Control annotation overlap:

# Each instance gets exactly 2 annotations
annotation_per_instance: 2
 
# Assign same instances to specific users for IAA calculation
overlap_users:
  - user1@example.com
  - user2@example.com
overlap_percentage: 20  # 20% of instances shared

Quality Control

Attention Checks

Insert test questions to verify attention:

attention_checks:
  enabled: true
  frequency: 10  # Every 10 instances
  fail_threshold: 2  # Max failures before warning

Qualification Tests

Require passing a test before main annotation:

qualification:
  enabled: true
  test_file: qualification_test.json
  min_score: 80  # Minimum percentage to pass

Session Management

Session Timeout

Auto-logout after inactivity:

session_timeout: 3600  # seconds (1 hour)

Save Progress

Annotations are saved automatically, but you can configure:

auto_save: true
auto_save_interval: 30  # seconds

User Statistics

Track annotator performance:

track_user_stats: true

Available metrics:

  • Annotations completed
  • Time per annotation
  • Agreement with others
  • Attention check performance

Access via admin dashboard at /admin.

Multi-Annotator Workflows

Round-Robin Assignment

Distribute instances evenly:

assignment_strategy: round_robin

Priority-Based Assignment

Assign based on annotator expertise:

assignment_strategy: priority
user_priorities:
  expert@example.com: high
  novice@example.com: low

Privacy Settings

Anonymize Users

Hide user identities in exports:

anonymize_users: true

Data Retention

Configure how long to keep data:

data_retention:
  annotations: 365  # days
  user_data: 90     # days

Example: Research Team Setup

# Restricted to research team
allow_all_users: false
 
# Team members
authorized_users:
  - researcher1@university.edu
  - researcher2@university.edu
  - student1@university.edu
  - student2@university.edu
 
# Lead researcher is admin
admin_users:
  - researcher1@university.edu
 
# Each person annotates 200 items
instances_per_annotator: 200
 
# Each item gets 2 annotations for reliability
annotation_per_instance: 2
 
# Track performance
track_user_stats: true
 
# Auto-save every 30 seconds
auto_save: true
auto_save_interval: 30

Example: Crowdsourcing Setup

# Open access for crowdworkers
allow_all_users: true
 
# Prolific integration
prolific_integration: true
prolific_completion_code: "POTATO2024"
 
# Limit per worker
instances_per_annotator: 50
 
# Quality control
attention_checks:
  enabled: true
  frequency: 10
 
# Multiple annotations per item
annotation_per_instance: 3
PreviousAnnotation Schemes