# How to Let a Coding Agent Write Your Annotation Config

Source: https://www.potatoannotator.com/docs/guides/configs-your-coding-agent-can-check

**The most common failure when an LLM writes an annotation config is inventing an option that does not exist.** The model produces something plausible, the server ignores the unknown key or rejects the file, and the error surfaces minutes later as behaviour nobody configured. A machine-readable schema moves that failure to the moment of writing.

## Add the modeline

One line at the top of a config:

```yaml
# yaml-language-server: $schema=https://potatoannotator.readthedocs.io/en/latest/schemas/potato-config.schema.json
```

That turns on live validation in VS Code, JetBrains IDEs, Zed and Helix. An invented `annotation_type` is underlined as you type, and completion offers the real ones.

Potato ships this line on every example config, and its [playground](/playground) emits it on every download.

## Validate in CI

```bash
check-jsonschema --schemafile \
  https://potatoannotator.readthedocs.io/en/latest/schemas/potato-config.schema.json \
  config.yaml
```

The JSON Schema is deliberately permissive about unknown keys, because the server only warns on them, and a stricter schema would reject working configs. So pair it with the project's own validator, which reports them:

```bash
python -m potato.validate_cli --strict config.yaml
```

`--strict` treats an unknown key as fatal. That is what catches a typo like `annotaton_type`, which a permissive schema accepts silently.

## Give the agent the spec

If you are pointing Claude Code, Codex or Cursor at a tool, generated specs are worth more than prose documentation, because they describe what the code does today rather than what someone remembered:

| Artifact | Covers |
|---|---|
| [Config JSON Schema](https://potatoannotator.readthedocs.io/en/latest/schemas/potato-config.schema.json) | 159 config keys, 61 annotation types, 24 display types, plus per-type conditionals |
| [OpenAPI 3.1](https://github.com/davidjurgens/potato/blob/main/docs/api-reference/openapi.json) | 419 paths, 456 operations |
| `llms.txt` | Curated documentation index |

The per-type conditionals matter more than the key list: they are what tells an agent that a `constant_sum` scheme needs `labels`, before the server does.

## Generate the spec from the code

If you maintain the tool rather than use it, the property that makes this work is that the schema is **generated from the same registries the server validates against**, and CI fails when it drifts.

A hand-maintained schema is worse than none. It will be trusted, and it will be wrong the first time someone adds a type without updating it.

## What we do on this site

This site's own [config builder](/playground) is built the same way. It reads Potato's registries into a committed spec file, maps every field against it, and drops anything the registry does not contain. A harness then puts every generated config — one per annotation type, plus every starter template — through `validate_cli --strict`.

That harness caught real bugs: an annotation type that did not exist, three schemes missing required fields, and a duplicated YAML key that silently discarded the authorized-user list. None of them was visible by reading the output.

## Further reading

- [Machine-readable specs](/docs/measurement/machine-readable-specs)
- [Playground](/playground)
- [Configuration basics](/docs/getting-started/configuration-basics)