potato-skill: Quick Start with a Coding Agent
Describe what you need labeled to Claude Code, Codex, or Cursor, and potato-skill designs the annotation study, builds it in Potato, and checks that it renders.
You can start a Potato project in two ways. Write the task yourself, as the Quick Start shows, or describe the study to a coding agent and let potato-skill design and build it. This page covers the second route. Both end in the same place, a Potato annotation task you can run, change, and share.
What the skill decides
potato-skill is an agent skill, a package of instructions and helper scripts that a coding agent loads when a request needs them. It ships 28 reference files and nine helper scripts, and it covers the design of the study as well as the build:
- what to ask annotators, and on what scale
- how many annotators each item needs
- how to train annotators and check their work
- the interface annotators see
- exporting the results in the format the next step needs
Some of those it settles on its own, and some it hands back. The rule it works to is that a decision is made locally when being wrong is cheap to fix, and handed back when being wrong is expensive, irreversible, or not the agent's to make. It takes a 5-point scale with every point labeled, keyboard shortcuts on any question with nine labels or fewer, random item order with a fixed random_seed, and all questions required. It asks you about the unit of annotation, the number of annotators per item when agreement is going to be reported, the consent wording, and any label set with no "unclear" option.
The task it builds runs on Potato, so it gets what Potato already does: a login for each annotator, several annotators per item, agreement statistics, Prolific and MTurk integration, and export to the formats downstream tools read.
Requirements
Potato has to be installed where the agent runs commands, because the skill's helper scripts import Potato's registries and call the potato command:
pip install potato-annotationYou also need a coding agent. The skill was written for Claude Code, and it installs into Codex and Cursor as well.
Installing the skill
In Claude Code, install it from the plugin marketplace:
/plugin marketplace add davidjurgens/potato-skill
/plugin install potato-skill@potatoFor Codex and Cursor, the skills CLI installs it into the project you are working in:
npx skills add davidjurgens/potato-skill --agent codex cursorWhat a Codex or Cursor install includes
The skills CLI writes .agents/skills/potato-skill/ with SKILL.md, all 28 references, and all nine scripts, so a Codex or Cursor project gets the same material a Claude Code install gets. It leaves AGENTS.md and .cursor/rules untouched, and how each tool discovers a skill in that directory is up to the tool.
The repository also carries an AGENTS.md at its root, which Codex and Cursor read by filename with no install step. That file is a short summary against roughly 79,000 words across the references, and it points at reference files and scripts that only a full install puts on disk. Use it when you want an agent to have the check-before-handoff loop and the rules that bite, and use the skills CLI when you want the reference material behind them.
Describing the study
Say what needs labeling, who will label it, and what the labels are for. You do not need to know Potato's annotation types or its config format. A request can be this short:
I want 200 tweets labeled for sentiment on a 5-point scale by three annotators each, with a consent page and two attention checks. Build it and start it.
The skill works out the question wording, the scale, how items are assigned, and how annotators are checked. It does not block on the questions it wants confirmed. It states its assumptions, builds the task, verifies it, and asks alongside something you can already click through.
Checks the agent runs before handing it back
A config that validates tells you the server will start, and tells you nothing about whether a person can do the work. The skill instructs the agent to render the task and look at it:
potato validate config.yaml --strict
potato preview config.yaml --screenshot shot-01.png--strict turns an unrecognized config key from a warning into a failure, which catches a typo that would otherwise leave a feature silently off. --screenshot boots a real Potato server and drives it in headless Playwright, recording console errors, uncaught page exceptions, and HTTP responses of 400 or above. It exits 0 only when all three lists are empty.
Three of the nine helpers automate what can be measured rather than judged. check_ui.py measures the live layout and reports schemes or a Next button below the fold, empty media widgets, and duplicate shortcuts. boot_and_check.py reports every feature that is configured but loaded nothing. walk_task.py walks a running task the way an annotator would and reports where it stops.
Open the task in your browser and label a few items yourself before you invite anyone.
The task it produces
A Potato task is a project folder holding the task's config, its data, and its instructions. Everything in the Quick Start applies to it. You start it with potato start config.yaml, edit it by hand, add annotators, and export the labels.
Because the result is a Potato task folder, other people can use it too. A colleague can run the same study on new data, you can rerun it next year, and you can publish it with the paper that used it. The showcase is a collection of tasks shared this way, taken from published papers.
Existing designs and annotations
- A published design. The skill can start from one of 440 annotation designs taken from published papers, with their label sets, question wording, and instructions. You can browse them in the showcase, and
find_design.pysearches them for a design close to the one you described. - Another tool's annotations.
potato importreads 20 formats, including Labelbox, CVAT, and COCO, so a half-finished project can carry on in Potato. - A study that is already running.
study_status.pyreports progress for each annotator and agreement for each question, and the skill says which fixes are safe mid-study and which would corrupt the data already collected.
Potato's MCP servers
An agent can reach the same answers over MCP instead of the shell. potato mcp serve --root . exposes 12 authoring tools that read the same registries as the CLI, including render_task_screenshot, which returns the rendered page as an image. potato mcp connect bridges a running study and adds 12 live tools covering status, progress, annotators, agreement, assignment, and export. The bridge needs an mcp block in the config listing the tools to expose, and a token from potato mcp issue-token.
Tests against Potato's registries
A plausible wrong identifier is worse than no documentation, because an agent will use it. Three of the 28 references are generated from Potato's own registries and cannot drift from what the server enforces: all 61 annotation types, each with a worked example lifted from a real project, the documented top-level config keys, and the documented sub-keys.
CI covers the skill's core files, checking that every annotation type, display type, config key, operator, strategy name, and command named in them exists in Potato. Every YAML sample is spliced into a working config and run through Potato's real validator, and the worked example boots a real server.
Its reference material is published at davidjurgens.github.io/potato-skill, readable without installing anything. Anthropic recommends reviewing any skill before you install it.
Comparing the two routes
The two routes differ in what you start from, not in what you end up with:
| Write the task yourself | Describe it to a coding agent | |
|---|---|---|
| You start with | a YAML config | a description in plain language |
| Suits | learning Potato, small tasks | new studies, crowdsourced studies with training and checks, annotation types you have not used |
| You end with | a Potato task | a Potato task |
You can switch routes at any point. An agent can change a task you wrote, and you can edit one the agent built.
Questions
The questions below cover setting up potato-skill with a coding agent and finding its source.
How do I use Potato with a coding agent?
Install Potato and potato-skill where the agent runs commands, then describe the study to the agent in plain language. pip install potato-annotation installs Potato. In Claude Code, /plugin marketplace add davidjurgens/potato-skill followed by /plugin install potato-skill@potato installs the skill, and in Codex or Cursor, npx skills add davidjurgens/potato-skill --agent codex cursor does. Tell the agent what needs labeling, who will label it, and what the labels are for. potato-skill builds the task, renders it to check that it works, and hands back a list of the decisions it made for you.
Where is potato-skill on GitHub?
potato-skill is at github.com/davidjurgens/potato-skill, released under the GPL-3.0-or-later. Its reference material is published at davidjurgens.github.io/potato-skill, so you can read what the skill tells the agent before you install it.