Admin Dashboard

Potato 2.0 includes an admin dashboard for monitoring annotation progress, tracking annotator performance, and managing task configuration in real time.

Accessing the Dashboard

API Key Authentication

Configure an admin API key in your configuration:

admin_api_key: "your-secure-api-key-here"

Access the dashboard at /admin and enter your API key when prompted.

Authentication Methods

1. Direct navigation - Go to /admin and enter the API key
2. Header-based - Include X-API-Key header in requests
3. Debug mode - Set debug: true for development (not recommended for production)

# Development only
debug: true  # Bypasses authentication

Dashboard Tabs

Overview Tab

High-level metrics for your annotation task:

• Total Users - Number of registered annotators
• Total Annotations - Total annotations completed
• Completion Rate - Percentage of data annotated
• Active Sessions - Currently active annotators
• Task Configuration - Current settings summary

Annotators Tab

Detailed information about each annotator:

Features:

• Sort by any column
• Filter by phase or activity
• Export annotator data
• View individual annotator details

Instances Tab

Browse and manage annotation instances:

Features:

• Pagination (25, 50, or 100 per page)
• Sort by annotations, disagreement, etc.
• Filter by annotation count
• View full instance details

Configuration Tab

Modify settings in real-time without restarting:

• Max Annotations per User - Limit how many items each user can annotate
• Max Annotations per Item - Target annotations per instance
• Assignment Strategy - How instances are assigned to users

Changes take effect immediately.

Assignment Strategies

Configure how instances are assigned from the dashboard:

Timing Analytics

The dashboard tracks detailed timing data:

• Total Working Time - Sum of all annotation sessions
• Average Time per Annotation - Mean time per item
• Annotations per Hour - Throughput rate
• Session Duration - Time per login session

Timing data is derived from behavioral tracking and page interaction events.

Configuration

Basic Setup

# Enable admin dashboard
admin_api_key: "your-secure-api-key"
 
# Optional: customize admin settings
admin:
  session_timeout: 3600  # seconds
  max_export_size: 10000  # rows

Full Configuration

admin_api_key: "your-secure-api-key"
 
admin:
  # Session settings
  session_timeout: 3600
 
  # Export limits
  max_export_size: 10000
 
  # Dashboard refresh
  auto_refresh: true
  refresh_interval: 30  # seconds
 
  # Pagination defaults
  default_page_size: 50

API Endpoints

The admin dashboard is powered by REST API endpoints:

Stats

GET /api/admin/stats

Returns overall task statistics.

Annotators

GET /api/admin/annotators
GET /api/admin/annotators/{user_id}

List all annotators or get details for a specific user.

Instances

GET /api/admin/instances
GET /api/admin/instances/{instance_id}

List instances with pagination and filtering.

Configuration

GET /api/admin/config
POST /api/admin/config

Get or update task configuration.

Training Progress

GET /api/admin/training/stats
GET /api/admin/training/user/{user_id}

Training phase statistics and per-user progress.

Active Learning

GET /api/admin/active-learning/status
GET /api/admin/active-learning/metrics

Active learning model status and performance metrics.

Monitoring Features

Suspicious Activity Detection

The dashboard can flag potentially problematic behavior:

• Very fast annotation speeds
• Unusual session patterns
• Consistent answer patterns
• Missing annotations

Quality Metrics

Track annotation quality indicators:

• Inter-annotator agreement
• Time-per-annotation distribution
• Label distribution balance
• Training phase performance

Security Best Practices

API Key Management

# Use environment variables
admin_api_key: ${ADMIN_API_KEY}

export ADMIN_API_KEY="your-secure-key"

Access Control

• Use strong, unique API keys
• Rotate keys periodically
• Don't expose admin endpoints publicly
• Use HTTPS in production

Audit Logging

The dashboard tracks admin actions:

• Configuration changes
• User management actions
• Data exports
• Access attempts

Example Configuration

task_name: "Sentiment Analysis"
task_dir: "."
port: 8000
 
# Admin dashboard
admin_api_key: ${ADMIN_API_KEY}
 
admin:
  session_timeout: 7200
  auto_refresh: true
  refresh_interval: 60
  default_page_size: 50
 
# Data and annotation configuration
data_files:
  - "data/reviews.json"
 
item_properties:
  id_key: id
  text_key: text
 
annotation_schemes:
  - annotation_type: radio
    name: sentiment
    labels:
      - Positive
      - Negative
      - Neutral
 
# Assignment settings (editable from dashboard)
assignment:
  strategy: least_annotated
  max_annotations_per_user: 100
  max_annotations_per_item: 3
 
output_annotation_dir: "output/"
output_annotation_format: "json"
allow_all_users: true

Performance Considerations

Large Datasets

For datasets with thousands of instances:

• Use pagination (instances tab)
• Enable lazy loading
• Consider database backend for better performance

Many Annotators

For tasks with many concurrent annotators:

• Set appropriate refresh intervals
• Monitor server resources
• Use database backend for scalability

# For large-scale deployments
database:
  type: mysql
  host: localhost
  database: potato_db
  user: ${DB_USER}
  password: ${DB_PASSWORD}

Troubleshooting

Can't Access Dashboard

• Verify API key is set in config
• Check URL is /admin
• Ensure API key matches exactly
• Check for typos in configuration

Slow Dashboard Loading

• Reduce page size
• Increase refresh interval
• Consider database backend
• Check server resources

Stats Not Updating

• Verify auto-refresh is enabled
• Check refresh interval setting
• Manually refresh the page
• Check for JavaScript errors

Further Reading

• Behavioral Tracking - Detailed interaction tracking
• Quality Control - Attention checks and gold standards
• API Overview - Admin API endpoints

For implementation details, see the source documentation.