# Self-Hosted Annotation Tools: What Each Needs to Run and Where to Host It

Source: https://www.potatoannotator.com/blog/self-hosted-annotation-tools

**A self-hosted annotation tool needs four things from the server it runs on: a disk that survives restarts, a database the tool's own docs recommend for production, a way for annotators to sign in, and a backup you have restored at least once. The open-source tools differ most in how many services they bring with them, from one container to eighteen, and that number drives most of the hosting work a study takes on.**

*Self-hosting* here means running the annotation tool on a machine you control, whether a lab server, a university cloud allocation or a rented VM, so that the data never passes through the vendor's service. Self-hosting is the usual answer when an ethics board, a data-use agreement or a hospital will not let the data leave. The comparison below covers what a self-hosted install of six open-source tools requires, according to each project's documentation as of October 2026, and then compares hosts by cost and by what they keep across a restart.

## Requirements of a self-hosted install

Every tool below stores three kinds of state: the items to annotate (e.g., text, images, media), the annotations, and the accounts and projects that tie them together. A self-hosted install has to keep all three on a disk that survives a restart, back them up somewhere else, and put a sign-in in front of them. Two further choices follow from the tool's architecture. A tool that ships with SQLite usually recommends a database server for production, and a tool that ships with a queue, a cache or a search engine needs each of those running and persisted too. Most of the hosting work comes from those extra services. Each one is another container to keep healthy and another volume to back up, and its version has to stay compatible with the others on every upgrade.

## Six tools compared

The table summarizes what each project's documentation says about a self-hosted production install. "Services" counts the containers in the project's reference Docker Compose file.

| Tool | Supported shapes | Services | Database | SSO in the free edition | Server backup guide |
|---|---|---|---|---|---|
| Label Studio | pip, Docker, Compose, Helm | 3 | SQLite by default, PostgreSQL recommended at scale | SAML SSO and LDAP are Enterprise only | Enterprise docs only |
| CVAT | Compose, Helm | 18 | PostgreSQL | LDAP documented for Community; OIDC and SAML SSO are Enterprise | yes |
| doccano | pip, Docker, Compose | 6 | SQLite by default, PostgreSQL in the production Compose | GitHub and Okta OAuth | not found |
| Argilla | Hugging Face Spaces, Compose | 5 | SQLite by default, PostgreSQL supported, plus Elasticsearch or OpenSearch | OAuth2 and Keycloak | dataset export only |
| INCEpTION | JAR, Docker, Compose | 2 | embedded HSQLDB, MariaDB recommended for production | SAML2 and OIDC | yes |
| Potato | pip, Docker, `potato deploy` | 1 | files and SQLite, MySQL optional | OAuth and OIDC providers | built-in backup to HF or S3 |

**[Label Studio](https://labelstud.io/guide/install)** runs as one container or as a Compose stack of nginx, the app and PostgreSQL. It uses SQLite by default, and its [storage guide](https://labelstud.io/guide/storedata) says to consider PostgreSQL above 100,000 tasks with 5 or more concurrent users. The install guide asks for at least 8 GB of RAM, recommends 16 GB, and recommends 50 GB of disk for production. The [edition comparison](https://labelstud.io/guide/label_studio_compare) lists SAML SSO, LDAP, role-based access and activity logs as paid features, and in the Community edition every user can see every project.

**[CVAT](https://docs.cvat.ai/docs/administration/community/basics/installation/)** runs as Docker Compose or on Kubernetes. Its reference Compose file defines 18 services, among them PostgreSQL, Redis, kvrocks for caching, eight background workers, Traefik, an Open Policy Agent service for authorization, and a ClickHouse and Grafana analytics stack. CVAT documents backup step by step. Its guide stops the containers, archives the database, media and analytics volumes, and warns that a restore needs exactly the same CVAT version. LDAP is documented for the Community edition, while [OIDC and SAML single sign-on](https://docs.cvat.ai/docs/account_management/sso/) are marked Enterprise.

**[doccano](https://doccano.github.io/doccano/install_and_upgrade_doccano/)** starts as one container with SQLite, and its production Compose file adds Celery, RabbitMQ, nginx and PostgreSQL. Its install guide warns that upgrading the package loses a SQLite database. It documents GitHub and Okta sign-in. As of October 2026 its newest release on PyPI is 1.8.4, from July 2023, and a 1.8.5 published on GitHub in January 2026 was yanked from PyPI.

**[Argilla](https://docs.argilla.io/latest/getting_started/how-to-deploy-argilla-with-docker/)** deploys on Hugging Face Spaces or with Docker Compose, where the server, a worker, PostgreSQL, Elasticsearch and Redis run together. Elasticsearch 8.5 or OpenSearch 2.4 is required, and the deployment guide notes that most deployment issues involve Elasticsearch. On a free Space without persistent storage, a restart loses datasets, workspaces and users. Its most recent release dates from March 2025.

**[INCEpTION](https://inception-project.github.io/releases/42.1/docs/admin-guide.html)** runs as a Java application or a container, with MariaDB as the recommended production database because its embedded HSQLDB can be corrupted when the machine crashes. Annotations and texts live in files under its home folder, and the admin guide covers system-level backup and restore in full, with a backup section that opens with the warning that "a backup which cannot be successfully restored is worthless".

**[Potato](/docs/deployment/production-setup)** runs as one process, from pip or from a single Docker image. Each annotator's work is stored in a file under the output directory, with project data in SQLite next to it, and MySQL is an option rather than a requirement. The process holds the item pool and the assignment queue in memory, so it runs with one gunicorn worker and more threads, and grows by moving to a bigger machine rather than by adding servers. Sign-in through Google, GitHub, Hugging Face or any OIDC provider is part of the open-source package, configured under [`authentication.providers`](/docs/deployment/sso-oauth).

Each tool's service count reflects the jobs it takes on. CVAT's eighteen services carry separate workers for import, export, chunking and quality reports, an analytics stack and policy-based authorization, which a large team running a long-lived installation can use. Argilla requires a search engine for its datasets. Potato keeps to one container, which fits a study that one lab sets up, runs for weeks and tears down.

## Hosts by cost and disk persistence

Any of these tools runs on a VM with Docker, and the main choice is whether the host's disk survives a restart. The costs below are the estimates Potato 2.10 prints for its `potato deploy` targets at their default sizes, and the same machines run the other tools at the same price, subject to each tool's memory needs.

| Host | Monthly cost | Disk survives a restart |
|---|---|---|
| AWS Lightsail, 2 GB | $12 | yes |
| Hetzner Cloud, 2 vCPU and 4 GB | about €6 | yes |
| Fly.io Machine with a 1 GB volume | about $6 | yes |
| Jetstream2, with an ACCESS allocation | free | yes |
| Render web service | free, or $7 plus disk | with a paid disk only |
| Heroku Basic dyno | $7 | no |
| Hugging Face Docker Space | a PRO or Team plan | no |

[Jetstream2](https://jetstream-cloud.org) is an NSF-funded research cloud, so a US researcher with an [ACCESS](https://access-ci.org) allocation pays nothing, and each instance gets a DNS name with an ordinary certificate. Lightsail's flat price includes the IPv4 address and a 60 GB disk, and it needs only `lightsail:*` plus `sts:GetCallerIdentity`, which suits a restricted university IAM role. A 2 GB Lightsail instance fits Potato, and Label Studio's stated 8 GB minimum needs a larger plan.

## Backups on hosts that wipe the disk

Heroku wipes the disk every day, and Render's free tier and Hugging Face Spaces lose it on restart. A study on one of those hosts needs a backup that also restores, or a returning annotator finds an empty task.

From version 2.10, Potato's `backup` block copies the annotation output and snapshots of the project databases to a Hugging Face dataset or an S3-compatible bucket every five minutes by default, and restores them when a server starts with an empty disk. The restore brings back the accounts with the annotations, so annotators sign in with the same passwords and continue where they stopped, and it never overwrites annotations already on the disk. Add this block to a config, with `HF_TOKEN` set in the server's environment and the `hosting` extra installed:

```yaml
backup:
  schedule_minutes: 5
  restore_on_boot: true
  sinks:
    - type: huggingface
      repo_id: lab/pilot-annotations
```

`potato deploy up` adds the same backup with `--backup hf` or `--backup s3`, and it will not deploy to Heroku or a Hugging Face Space without one unless `--demo` declares the annotations disposable. Render, Railway, Fly and ECS Express require one in every case, because the project reaches those hosts through the backup's storage. On hosts whose disk does persist, the backup still keeps a second copy off the server, which covers a deleted instance as well as a restart.

Whatever the tool, test the restore before the study starts. Start a fresh instance from the backup, sign in as an annotator, and check that the last annotation you made is there. For Potato, `potato deploy pull` copies everything a deployment has collected to your machine, and `potato deploy destroy` will not remove a deployment that has never been pulled, so run the pull first at the end of a study. [Running Potato Locally or in the Cloud](/docs/deployment/quick-deploy) has the command for each host.

## Further reading

- [Production Setup](/docs/deployment/production-setup), for gunicorn, Docker and the reverse proxy
- [SSO and OAuth](/docs/deployment/sso-oauth), for configuring sign-in providers
- [Annotation tools compared](/docs/guides/annotation-tools-compared), for the features rather than the hosting