# Chemistry Annotation Software for Structures and Spectra

Source: https://www.potatoannotator.com/use-cases/chemistry

**Potato gives chemists a structure editor and a spectrum viewer in the browser, where annotators correct structures recognized from patents and papers, draw structures graded against an answer key, mark atoms such as reaction centers, and pick and integrate peaks on NMR, IR, and mass spectra. Potato is free and open source, runs on your own server, and exports structures to an SD file and peaks to a CSV file under each annotator's name.**

[Optical chemical structure recognition](https://en.wikipedia.org/wiki/Optical_chemical_structure_recognition) (OCSR) converts a drawing of a molecule in a patent or paper into a machine-readable structure. The OCSR model MolScribe reached 76% to 93% accuracy on public benchmarks ([Qian et al., 2023](https://doi.org/10.1021/acs.jcim.2c01480)), and a dataset built from patents therefore needs a person to check every recognized structure against the figure. In online chemistry teaching, most questions about chemical structures cannot be graded on a true/false basis, and a strict comparison with the expected structure fails when some tolerance is acceptable ([Plyer et al., 2022](https://doi.org/10.1186/s13321-022-00645-0)). Potato grades each structure as right or wrong, and an answer key can ignore stereochemistry or require only a substructure. Potato runs no OCSR model, and a study loads the recognized structures and processed spectra that other tools produce.

## Chemistry tasks in Potato

The `chemical_structure` and `peak_picking` schemes and the `molecule` and `spectrum` displays are new in Potato 2.13, and the linked pages have the configuration.

| Task | How Potato handles the task | Docs |
|---|---|---|
| Correct OCSR output | The editor opens on the recognized structure beside the source figure, and annotators fix the misread atoms, bonds, and stereocenters | [Chemical structures](/docs/annotation-types/chemical-structures) |
| Grade drawn structures | Students draw with the mouse or type a SMILES string, and Potato grades each drawing on the server against a SMILES, molfile, InChIKey, or SMARTS answer key | [Chemical structures](/docs/annotation-types/chemical-structures#answer-keys) |
| Mark reaction centers or sites of metabolism | A `molecule` display with a `viewer_selection` scheme stores each selected atom by index, beside an optional fixed highlight of a model's predicted atoms | [Chemical structures](/docs/annotation-types/chemical-structures#marking-atoms-on-a-molecule) |
| Pick and integrate NMR peaks | A click picks a peak, a drag in Integrate mode integrates a multiplet, and each peak takes fields such as an assignment and a multiplicity | [Spectra](/docs/annotation-types/spectra) |
| Review an automatic peak picker | The picker's peaks load as suggestions, and annotators accept or reject each suggested peak | [Spectra](/docs/annotation-types/spectra#reviewing-an-automatic-peak-picker) |
| Label IR and mass spectra | Minima on IR transmittance spectra, and sticks for centroided mzML scans and JCAMP-DX peak tables | [Spectra](/docs/annotation-types/spectra#mass-spectra-and-ir-spectra) |

## A structure correction study with two annotators

The config below has two annotators check every structure recognized from a patent figure, and each annotator marks misread atoms, corrects the structure, and rates the recognition. The [chemical structures](/docs/annotation-types/chemical-structures) page has every option and the item format.

```yaml
annotation_task_name: "Patent structure check"
task_dir: .
media_directory: media
output_annotation_dir: annotation_output
data_files: [data/structures.json]  # each item: figure, OCSR SMILES, true SMILES
item_properties: {id_key: id, text_key: note}
user_config:
  allow_all_users: true

num_annotators_per_item: 2

instance_display:
  layout:
    direction: horizontal
  fields:
    - key: page_image             # the figure cropped from the patent
      type: image
      label: "Source image"
    - key: ocsr_smiles            # what the OCSR tool produced
      type: molecule
      label: "Recognized structure"

annotation_schemes:
  - annotation_type: viewer_selection
    name: wrong_atoms
    description: "Mark the atoms the recognizer got wrong"
    viewer_field: ocsr_smiles
    labels: ["wrong element", "wrong bond or charge", "wrong stereo"]

  - annotation_type: chemical_structure
    name: corrected
    description: "Correct the structure so it matches the source image"
    prefill_field: ocsr_smiles
    answer_field: true_smiles     # graded in the export, never sent to the page
    rdkit: false                  # OpenChemLib only, graded by the chemistry extra
    label_requirement: {required: true}

  - annotation_type: radio
    name: verdict
    description: "How good was the recognition?"
    labels: ["Correct", "Minor errors", "Wrong structure"]
    label_requirement: {required: true}
```

`num_annotators_per_item: 2` sends each structure to two annotators. Potato removes `true_smiles` from everything the page receives. With `rdkit: false`, the browser loads OpenChemLib only, and the `chemistry` extra (`pip install 'potato-annotation[chemistry]'`) grades each corrected structure at export. Comparing `answer_correct` with `changed` in the export separates the errors an annotator missed from the errors an annotator introduced.

## Getting structures and spectra in and answers out

A structure field holds a SMILES string, molfile text, or a `.mol`, `.sdf`, or `.smi` file name. Spectra are read on the Potato server from the media directory, and the file extension picks the reader:

- **JCAMP-DX.** `.jdx`, `.dx`, and `.jcamp` files open with no extra install, including compressed `##XYDATA`, peak tables, and the `##NTUPLES` form that Bruker and MestReNova write.
- **mzML.** Mass spectrometry runs open with the `spectra` extra (`pip install 'potato-annotation[spectra]'`), and an item can name one scan of the run.
- **CSV and text.** Two columns, x then y, with optional units in a header row.

The `chemistry` export writes `chemical_structures.csv` with one row per answer, `chemical_structures.sdf` with each answer's molfile, and `molecule_atoms.csv` with one row per selected atom. The `peak_table` export writes `peaks.csv` and `peaks.jsonl`, with one row per peak, integrated region, and rejected suggestion:

```bash
python -m potato.export --config config.yaml --format chemistry --output out/
python -m potato.export --config config.yaml --format peak_table --output out/
```

Potato's agreement metrics do not cover the `chemical_structure`, `viewer_selection`, or `peak_picking` schemes. To compare two annotators' structures, match the rows of `chemical_structures.csv` on `instance_id` and `schema` and compare `inchikey`. For peaks, match each item's rows in `peaks.csv` within a tolerance in ppm or cm⁻¹, and the counts of accepted and rejected suggestions give each annotator's acceptance rate for a peak picker.

## Working alongside ChemDraw, RDKit, and Mnova

[ChemDraw](https://revvitysignals.com/products/research/chemdraw) from Revvity Signals is commercial drawing software that universities such as Stanford license campus-wide, and suits drawing structures for papers and theses. [RDKit](https://www.rdkit.org/) is open-source cheminformatics software for C++ and Python, and suits scripted analysis of the exported structures. [Mnova NMR](https://mestrelab.com/main-product/nmr) from Mestrelab processes and analyzes 1D and 2D NMR data from many spectrometer vendors, and suits processing raw data into a spectrum. [NMRium](https://www.nmrium.com/) processes NMR spectra in the browser from JCAMP-DX files, Bruker folders, and JEOL files, and offers structure elucidation exercises for teaching. Potato fits when several annotators answer the same items, because Potato assigns each structure or spectrum to a set number of annotators, trains annotators on graded practice items, and exports each annotator's answers separately. One route is to process a spectrum in Mnova, save the spectrum as JCAMP-DX, and have annotators pick the peaks in Potato.

## Setting up the study with a coding agent

[potato-skill](/docs/getting-started/quick-start-with-a-coding-agent) 1.2.0 knows `chemical_structure`, `molecule`, `peak_picking`, and `spectrum` by name and guides Claude Code, Codex, or Cursor through the study-design side, such as what to ask annotators, how many annotators see each item, training and attention checks, and export. The skill does not yet document the structure and spectrum formats, the answer-key modes, the display options, or the `chemistry` and `spectra` extras. Point the agent at the [chemical structures](/docs/annotation-types/chemical-structures) and [spectra](/docs/annotation-types/spectra) pages, or at the pages' Markdown copies such as `https://www.potatoannotator.com/docs/annotation-types/spectra.md`, for the formats and options.

## Frequently asked questions

### Can annotators check DECIMER or MolScribe output against the patent figure?

Yes. Each item gives the figure cropped from the patent and the SMILES string the OCSR tool produced, and the editor opens on the recognized structure beside the figure. Annotators mark the misread atoms and correct the structure. A structure left unchanged is saved with `changed: false`, and `answer_correct` in the export says whether each saved structure matches the true structure.

### Does the answer key tell stereoisomers apart?

Yes, by default. The exact mode compares standard InChIKeys, and a drawing with the wrong configuration at a stereocenter fails against the key. Adding `ignore_stereo: true` to a structure key compares only the first 14-character block of the InChIKey, and any stereoisomer of the expected structure then passes. The first block encodes connectivity alone.

### Can Potato read JCAMP-DX files with compressed XYDATA?

Yes. Potato reads `##XYDATA` in every ASDF form, including SQZ, DIF, and DUP compression, and checks the DIF Y-check value at the start of each continuation line, refusing a file whose check fails as corrupt. 2D NMR, JCAMP-CS structures, and FIDs without an abscissa are not read.

## Further reading

- [Chemical structures](/docs/annotation-types/chemical-structures), for the editor, the answer keys, and the export files
- [Spectra](/docs/annotation-types/spectra), for every spectrum format, display option, and export column
- [Training phase](/docs/features/training-phase) and [quality control](/docs/features/quality-control), for graded practice items and gold-standard structures
- [How many annotators do you need?](/docs/guides/how-many-annotators), for choosing the number of annotators per item
- [Life sciences](/use-cases/life-sciences), for variant review, protein structures, and biomedical text on this site

## References

1. Qian et al. (2023). MolScribe: Robust Molecular Structure Recognition with Image-to-Graph Generation. *Journal of Chemical Information and Modeling* 63(7). https://doi.org/10.1021/acs.jcim.2c01480
2. Plyer et al. (2022). Implementation of a soft grading system for chemistry in a Moodle plugin. *Journal of Cheminformatics* 14. https://doi.org/10.1186/s13321-022-00645-0