Admin Dashboard

Potato 2.0 includes a comprehensive admin dashboard for monitoring annotation progress, tracking annotator performance, and managing task configuration in real-time.

Accessing the Dashboard

API Key Authentication

Configure an admin API key in your configuration:

admin_api_key: "your-secure-api-key-here"

Access the dashboard at /admin and enter your API key when prompted.

Authentication Methods

1. Direct navigation - Go to /admin and enter the API key
2. Header-based - Include X-API-Key header in requests
3. Debug mode - Set debug: true for development (not recommended for production)

# Development only
debug: true  # Bypasses authentication

Dashboard Tabs

Overview Tab

High-level metrics for your annotation task:

• Total Users - Number of registered annotators
• Total Annotations - Total annotations completed
• Completion Rate - Percentage of data annotated
• Active Sessions - Currently active annotators
• Task Configuration - Current settings summary

Annotators Tab

Detailed information about each annotator:

Features:

• Sort by any column
• Filter by phase or activity
• Export annotator data
• View individual annotator details

Instances Tab

Browse and manage annotation instances:

Features:

• Pagination (25, 50, or 100 per page)
• Sort by annotations, disagreement, etc.
• Filter by annotation count
• View full instance details

Configuration Tab

Modify settings in real-time without restarting:

• Max Annotations per User - Limit how many items each user can annotate
• Max Annotations per Item - Target annotations per instance
• Assignment Strategy - How instances are assigned to users

Changes take effect immediately.

Assignment Strategies

Configure how instances are assigned from the dashboard:

Timing Analytics

The dashboard tracks detailed timing data:

• Total Working Time - Sum of all annotation sessions
• Average Time per Annotation - Mean time per item
• Annotations per Hour - Throughput rate
• Session Duration - Time per login session

Timing data is derived from behavioral tracking and page interaction events.

Configuration

Basic Setup

# Enable admin dashboard
admin_api_key: "your-secure-api-key"
 
# Optional: customize admin settings
admin:
  session_timeout: 3600  # seconds
  max_export_size: 10000  # rows

Full Configuration

admin_api_key: "your-secure-api-key"
 
admin:
  # Session settings
  session_timeout: 3600
 
  # Export limits
  max_export_size: 10000
 
  # Dashboard refresh
  auto_refresh: true
  refresh_interval: 30  # seconds
 
  # Pagination defaults
  default_page_size: 50

API Endpoints

The admin dashboard is powered by REST API endpoints:

Stats

GET /api/admin/stats

Returns overall task statistics.

Annotators

GET /api/admin/annotators
GET /api/admin/annotators/{user_id}

List all annotators or get details for a specific user.

Instances

GET /api/admin/instances
GET /api/admin/instances/{instance_id}

List instances with pagination and filtering.

Configuration

GET /api/admin/config
POST /api/admin/config

Get or update task configuration.

Training Progress

GET /api/admin/training/stats
GET /api/admin/training/user/{user_id}

Training phase statistics and per-user progress.

Active Learning

GET /api/admin/active-learning/status
GET /api/admin/active-learning/metrics

Active learning model status and performance metrics.

Monitoring Features

Suspicious Activity Detection

The dashboard can flag potentially problematic behavior:

• Very fast annotation speeds
• Unusual session patterns
• Consistent answer patterns
• Missing annotations

Quality Metrics

Track annotation quality indicators:

• Inter-annotator agreement
• Time-per-annotation distribution
• Label distribution balance
• Training phase performance

Security Best Practices

API Key Management

# Use environment variables
admin_api_key: ${ADMIN_API_KEY}

export ADMIN_API_KEY="your-secure-key"

Access Control

• Use strong, unique API keys
• Rotate keys periodically
• Don't expose admin endpoints publicly
• Use HTTPS in production

Audit Logging

The dashboard tracks admin actions:

• Configuration changes
• User management actions
• Data exports
• Access attempts

Example Configuration

task_name: "Sentiment Analysis"
task_dir: "."
port: 8000
 
# Admin dashboard
admin_api_key: ${ADMIN_API_KEY}
 
admin:
  session_timeout: 7200
  auto_refresh: true
  refresh_interval: 60
  default_page_size: 50
 
# Data and annotation configuration
data_files:
  - "data/reviews.json"
 
item_properties:
  id_key: id
  text_key: text
 
annotation_schemes:
  - annotation_type: radio
    name: sentiment
    labels:
      - Positive
      - Negative
      - Neutral
 
# Assignment settings (editable from dashboard)
assignment:
  strategy: least_annotated
  max_annotations_per_user: 100
  max_annotations_per_item: 3
 
output_annotation_dir: "output/"
output_annotation_format: "json"
allow_all_users: true

Performance Considerations

Large Datasets

For datasets with thousands of instances:

• Use pagination (instances tab)
• Enable lazy loading
• Consider database backend for better performance

Many Annotators

For tasks with many concurrent annotators:

• Set appropriate refresh intervals
• Monitor server resources
• Use database backend for scalability

# For large-scale deployments
database:
  type: mysql
  host: localhost
  database: potato_db
  user: ${DB_USER}
  password: ${DB_PASSWORD}

Troubleshooting

Can't Access Dashboard

• Verify API key is set in config
• Check URL is /admin
• Ensure API key matches exactly
• Check for typos in configuration

Slow Dashboard Loading

• Reduce page size
• Increase refresh interval
• Consider database backend
• Check server resources

Stats Not Updating

• Verify auto-refresh is enabled
• Check refresh interval setting
• Manually refresh the page
• Check for JavaScript errors