# Chemical Structure Annotation and OCSR Correction

Source: https://www.potatoannotator.com/docs/annotation-types/chemical-structures

*New in v2.13.0*

The `chemical_structure` annotation scheme gives annotators a structure editor, and the `molecule` display draws a small molecule on which annotators select atoms. Chemistry dataset builders use the pair for structure correction. In structure correction, annotators compare a structure recognized from a figure in a patent or paper with the figure and fix the misread atoms and bonds. Chemistry education researchers and instructors use the editor to collect drawn structures, such as the structure a compound name describes, and Potato grades each drawing against an answer key on the server. Curators use the `molecule` display with a `viewer_selection` scheme to mark atoms, such as a reaction center or a site of metabolism. Each answer is stored as a molfile with the drawing's canonical SMILES, and the `chemistry` export writes every annotator's structures to a CSV file and an SD file.

[Optical chemical structure recognition](https://en.wikipedia.org/wiki/Optical_chemical_structure_recognition) (OCSR) converts an image of a chemical structure into a machine-readable structure. Potato runs no OCSR model, and a study loads the recognized structures produced by another tool. [SMILES](https://en.wikipedia.org/wiki/Simplified_Molecular_Input_Line_Entry_System) writes a structure as a short line of text, such as `CC(=O)O` for acetic acid. An MDL molfile is the connection-table text format of the [chemical table file](https://en.wikipedia.org/wiki/Chemical_table_file) family, and an SD file holds many molfiles with named data fields after each one. The [InChIKey](https://en.wikipedia.org/wiki/International_Chemical_Identifier) is a 27-character hash of the IUPAC InChI identifier, and two drawings of the same structure give the same InChIKey.

- **Editor:** [OpenChemLib](https://github.com/cheminfo/openchemlib-js) JS 9.25.0 (BSD-3-Clause), committed with Potato. The editor draws on a canvas with a toolbar for bonds, rings, elements, charges, stereo wedges, erasing, and undo. A text box under the canvas takes a typed or pasted SMILES string or molfile, and the text box is the keyboard route into the editor.
- **RDKit.js (optional):** the [RDKit](https://www.rdkit.org/) JavaScript MinimalLib 2026.3.6 (BSD-3-Clause) is a 7 MB WebAssembly build and is not committed. `potato vendor fetch rdkit` downloads the build and checks the download against the npm registry's integrity hash. With RDKit.js, the browser also computes the RDKit canonical SMILES, the InChI, and the InChIKey, flags structures RDKit cannot sanitize, and the `molecule` display can draw with `engine: rdkit`.
- **The `chemistry` extra (optional):** installs the `rdkit` Python package on the server. The package grades answer keys, and the `chemistry` export uses the package to fill in InChI and InChIKey for answers saved without RDKit.js.
- **No WebGL:** both toolkits draw in 2D.
- **Offline:** structures come from the item or from a file in the media directory, and answer keys never reach the browser.

The features available with each install are:

| Feature | OpenChemLib only | With RDKit.js | With the `chemistry` extra |
|---|---|---|---|
| Editor, molfile, OpenChemLib SMILES and ID code, formula, valence checks | yes | yes | yes |
| RDKit SMILES, InChI, and InChIKey in the stored answer | no | yes | filled in at export |
| RDKit sanitization check while drawing | no | yes | no |
| `molecule` display with `engine: rdkit` | no | yes | no |
| Grading against `smiles`, `molfile`, and SMARTS keys | no | no | yes |
| Grading against an `inchikey` key | no | yes, with the InChIKey from the browser | yes |

Server-side grading needs the extra, and RDKit.js is a separate fetch:

```bash
pip install 'potato-annotation[chemistry]'
potato vendor fetch rdkit     # optional, InChIKeys in the browser
```

Restart the server after the fetch. While RDKit.js is missing, `potato validate` warns for each `chemical_structure` scheme left at `rdkit: auto` and names the fetch command.

## Configuration

The config below sets up OCSR correction. Each item shows the source figure beside the recognized structure, and annotators mark the atoms the recognizer misread, correct the structure in the editor, and give a verdict:

```yaml
annotation_task_name: "Structure recognition check"
task_dir: .
media_directory: media
output_annotation_dir: annotation_output
data_files: [data/structures.json]
item_properties: {id_key: id, text_key: note}

instance_display:
  layout:
    direction: horizontal
  fields:
    - key: page_image             # the figure cropped from the patent or paper
      type: image
      label: "Source image"
    - key: ocsr_smiles            # what the OCSR tool produced
      type: molecule
      label: "Recognized structure"
      display_options:
        height: 300

user_config:
  allow_all_users: true

annotation_schemes:
  - annotation_type: viewer_selection
    name: wrong_atoms
    description: "Mark the atoms the recognizer got wrong"
    viewer_field: ocsr_smiles
    labels: ["wrong element", "wrong bond or charge", "wrong stereo"]

  - annotation_type: chemical_structure
    name: corrected
    description: "Correct the structure so it matches the source image"
    prefill_field: ocsr_smiles
    answer_field: true_smiles     # graded in the export, never sent to the page
    label_requirement: {required: true}

  - annotation_type: radio
    name: verdict
    description: "How good was the recognition?"
    labels: ["Correct", "Minor errors", "Wrong structure"]
    label_requirement: {required: true}
```

The editor starts from `ocsr_smiles`. A pre-filled structure that the annotator leaves as given becomes the saved answer with `changed: false`, and a correct recognition therefore needs no edit. The `verdict` question separates an accepted structure from an item the annotator never checked. Potato removes the `true_smiles` field from everything the page receives. In the export, `answer_correct` says whether each corrected structure matches `true_smiles`. Comparing `answer_correct` with `changed` separates missed errors (unchanged and wrong) from introduced errors (changed and wrong).

Each item names the figure relative to `media_directory` and gives both structures:

```json
{"id": "fig1_aspirin", "note": "Compound 1 as printed.", "page_image": "fig1_aspirin.png",
 "ocsr_smiles": "CC(=O)Oc1ccccc1C(=O)N", "true_smiles": "CC(=O)Oc1ccccc1C(=O)O"}
```

### Drawing a structure from a name

A name-to-structure exercise needs one scheme and an answer key per item. The config below shows the compound name as the item text:

```yaml
annotation_task_name: "Name to structure"
task_dir: .
output_annotation_dir: annotation_output
data_files: [data/names.json]
item_properties: {id_key: id, text_key: name}

user_config:
  allow_all_users: true

annotation_schemes:
  - annotation_type: chemical_structure
    name: drawn
    description: "Draw the structure this name describes"
    answer_field: expected
    rdkit: false                  # OpenChemLib only, no RDKit.js warning
    editor_height: 420
    label_requirement: {required: true}
```

```json
{"id": "q2", "name": "(S)-2-aminopropanoic acid", "expected": "C[C@H](N)C(=O)O"}
```

Students can draw with the mouse or type a SMILES string and press Enter. With `rdkit: false`, the page loads OpenChemLib only and stores no InChIKey, and the `chemistry` extra grades the drawings at export.

### Marking atoms on a molecule

Atom marking pairs a `molecule` display with a `viewer_selection` scheme. The config below asks curators to mark sites of metabolism and draws a model's predicted sites in amber for reference:

```yaml
annotation_task_name: "Sites of metabolism"
task_dir: .
output_annotation_dir: annotation_output
data_files: [data/molecules.json]
item_properties: {id_key: id, text_key: note}

instance_display:
  fields:
    - key: smiles
      type: molecule
      display_options:
        show_atom_numbers: true
        highlight_field: predicted_site   # a model's predicted atoms, drawn in amber

user_config:
  allow_all_users: true

annotation_schemes:
  - annotation_type: viewer_selection
    name: som
    description: "Mark each atom where the compound is metabolized"
    viewer_field: smiles
    labels: ["hydroxylation", "N-dealkylation", "glucuronidation"]
    min_selections: 1
    label_requirement: {required: true}
```

```json
{"id": "m1", "note": "Predicted sites in amber.", "smiles": "CC(=O)Nc1ccc(O)cc1", "predicted_site": [0, 8]}
```

The item field that `highlight_field` names holds 0-based atom indices, as a list or as text such as `"0, 8"`. The highlight is fixed, and curators select atoms independently of the highlight. With `min_selections` and `label_requirement` together, Next stays blocked until the curator has selected at least one atom.

## Structure data

A structure field holds a SMILES string, molfile text, or the name of a `.mol`, `.sdf`, `.smi`, or `.smiles` file in the media directory. The server reads the first record of an `.sdf` file and the first token of the first non-empty line of a `.smi` file. Structure text and structure files are limited to 1 MB. A value with spaces that is not a molfile is refused, because a SMILES string has no spaces. The display never fetches a URL, and Potato refuses a file path outside the media directory.

## Options

| `chemical_structure` option | Default | Meaning |
|---|---|---|
| `prefill_field` | none | Item field with a SMILES string, a molfile, or a structure file name. The editor starts from the structure |
| `prefill_format` | `auto` | `auto`, `smiles`, or `molfile`. Needs `prefill_field` |
| `answer_field` | none | Item field with an answer key, graded in the `chemistry` export. Must differ from `prefill_field`, and is removed from what the page receives |
| `rdkit` | `auto` | `auto` uses RDKit.js when installed. `true` requires RDKit.js, and `potato validate` and server start fail without the bundle. `false` loads OpenChemLib only and silences the warning |
| `require_valid` | `true` | A structure with a valence problem, or one RDKit.js cannot sanitize, leaves the item incomplete. Enforced when the scheme is required |
| `allow_empty` | `false` | Accepts an empty canvas as an answer |
| `editor_height` | `380` | Editor height in pixels, from 240 to 900 |
| `show_identifiers` | `true` | Shows the SMILES, formula, and InChIKey under the editor |
| `instructions` | none | Text under the question |
| `label_requirement` | none | `{required: true}` makes the scheme required |

| `molecule` display option | Default | Meaning |
|---|---|---|
| `engine` | `openchemlib` | `openchemlib` or `rdkit`. A config with `engine: rdkit` fails `potato validate` until `potato vendor fetch rdkit` has run |
| `height` | `320` | Drawing height, in pixels or a CSS length |
| `format` | `auto` | `auto`, `smiles`, or `molfile` |
| `show_atom_numbers` | `false` | Draws each atom's molfile number beside the atom |
| `atom_list` | `true` | Shows the list of atom buttons under the drawing, the keyboard route to selection |
| `highlight_field` | none | Item field with 0-based atom indices drawn with a fixed highlight |
| `highlight_color` | `#f59f00` | Hex color of the fixed highlight |
| `max_atoms` | `500` | Larger molecules are not drawn, and the display says so. From 1 to 5,000 |
| `allow_remote` | `false` | Accepted, with no effect on this display |

| `viewer_selection` option | Default | Meaning |
|---|---|---|
| `viewer_field` | required | The `key` of the `molecule` field in `instance_display` |
| `labels` | none | Names for selections, as strings or `{name, color}`. New selections take the current label, and each atom can be relabeled |
| `min_selections` | none | The item stays incomplete until this many atoms are selected, when the scheme is required |
| `max_selections` | none | Selecting more atoms is refused with a message |
| `instructions` | none | Text under the question |

## Answer keys

An answer key is a bare SMILES string, a molfile, or a mapping. All parts of a mapping must pass:

| Key | Passes when |
|---|---|
| `smiles`, `molfile`, or `inchikey` | The drawn structure's InChIKey equals the key's InChIKey. Give one of the three |
| `ignore_stereo: true` | Added to a structure key, compares only the first 14-character block of the InChIKey, which encodes connectivity. Stereo and isotope labels are not checked |
| `substructure` | The drawn structure contains the [SMARTS](https://en.wikipedia.org/wiki/SMILES_arbitrary_target_specification) pattern, or every pattern in a list |
| `not_substructure` | The drawn structure contains none of the SMARTS patterns |

The exact mode compares standard InChIKeys. Standard InChI treats some tautomers as one structure, and a drawing of 2-pyridone therefore passes against a key of 2-hydroxypyridine. Charges count, and ethoxide fails against ethanol even with `ignore_stereo`. A substructure key grades a feature of the answer, such as `{"substructure": "C(=O)[OH]"}` for a carboxylic acid.

An `answer_field` value may be a bare SMILES string. Gold-standard items and training items write the mapping form under the scheme's name:

```json
{"id": "gold_1", "gold_label": {"corrected": {"smiles": "CC(=O)Oc1ccccc1C(=O)O"}}}
{"id": "train_1", "correct_answers": {"drawn": {"substructure": "C(=O)[OH]"}}}
```

[Quality control](/docs/features/quality-control) and the [training phase](/docs/features/training-phase) grade these keys with the same comparison as the export. The editor reads `prefill_field` only in the annotation phase, and a training item therefore opens with an empty canvas.

Without the `rdkit` Python package, a `smiles`, `molfile`, or SMARTS key grades every answer wrong, and the server logs the missing package once. One case still works. An `inchikey` key matches an answer whose InChIKey RDKit.js computed in the annotator's browser. With the package installed, the server recomputes the InChIKey from the stored molfile and ignores the browser's value.

## Drawing and correcting structures

To add a bond, the annotator clicks an empty part of the canvas or drags from an atom, after choosing a bond type, ring, element, or charge in the toolbar on the left. Pointing at an atom and typing an element symbol (`o`, `n`, `s`) changes the element. The text box takes a SMILES string or molfile, and Load (or Enter, for a SMILES string) replaces the drawing. Text OpenChemLib cannot read is refused, and the drawing is kept. Clear empties the canvas, and Restore the given structure brings back the pre-filled structure.

The SMILES, formula, and InChIKey under the editor update while the annotator draws. Valence problems are listed there in the form "C (atom 2) has 5 bonds; it allows 4.", and with `require_valid` the problems keep a required item from being submitted.

## Selecting atoms

Annotators click an atom in the drawing to select the atom, and click again to remove the selection. In the atom list, Tab moves to the list, the arrow keys, Home, and End move between atoms, and Space or Enter toggles an atom. With `labels`, the annotator picks a label first, and each listed atom has a label menu and a Remove button. The page names atoms by element and molfile number, and `O4` is the fourth atom of the source.

A stored atom index is the 0-based position of the atom in the source as written, in the order of atoms in the SMILES string or the molfile's atom block. The molfile atom number is the index plus 1. The index equals RDKit's atom index for the same input read with `removeHs=False`, whichever engine draws the molecule. OpenChemLib folds explicit hydrogens in a SMILES string (`[H]C(Cl)(Br)F`) into the neighboring atom, and with `engine: openchemlib` such a SMILES string cannot be used for selection. The display says so and offers no atoms at all. Use a molfile or `engine: rdkit` when individual hydrogens must be selectable. Implicit hydrogens (`CCO`) are never selectable.

## Stored answer

A `chemical_structure` scheme stores one JSON object per item:

```json
{"molfile": "\nOCL MolfileCreator  2D\n\n  0  0  0 ... V3000\n...M  END\n",
 "smiles": "CC(Oc1ccccc1C(O)=O)=O",
 "idcode": "dklB@@QmR[fUxUZBBF@@",
 "formula": "C9H8O4",
 "rdkit_smiles": "CC(=O)Oc1ccccc1C(=O)O",
 "inchi": "InChI=1S/C9H8O4/c1-6(10)13-8-5-3-2-4-7(8)9(11)12/h2-5H,1H3,(H,11,12)",
 "inchikey": "BSYNRYMUTXBXSQ-UHFFFAOYSA-N",
 "changed": true,
 "problems": [],
 "toolkits": {"openchemlib": "9.25.0", "rdkit": "2026.03.6"}}
```

The `molfile` is V3000. `smiles` is OpenChemLib's canonical isomeric SMILES, and `idcode` is OpenChemLib's canonical ID code. `rdkit_smiles`, `inchi`, and `inchikey` are present only when RDKit.js ran. Canonical SMILES differ between toolkits, and structures should therefore be compared by InChIKey or ID code, never by `smiles` against `rdkit_smiles`. `changed` is `false` while the drawing has the pre-filled structure's ID code.

A `viewer_selection` scheme on a `molecule` display stores the selected atoms:

```json
{"viewer": "molecule", "items": [{"atom": 12, "element": "N", "label": "wrong element"}]}
```

## Export

The `chemistry` export covers `chemical_structure` schemes and `viewer_selection` schemes bound to a `molecule` display:

```bash
python -m potato.export --config config.yaml --format chemistry --output out/
```

| File | Contents |
|---|---|
| `chemical_structures.csv` | One row per answer, with `user_id, instance_id, schema, smiles, rdkit_smiles, inchi, inchikey, idcode, formula, changed, problems, identifiers_from, answer_correct, answer_reason` |
| `chemical_structures.sdf` | Each answer's molfile, with the row's non-empty fields as SD properties |
| `molecule_atoms.csv` | One row per selected atom, with `user_id, instance_id, schema, viewer_field, atom, number, element, label`, where `number` is the molfile number |

`identifiers_from` is `browser` when RDKit.js computed the InChI and InChIKey, `server` when the export computed both with the `rdkit` Python package, and `none` when neither was available. The export lists a warning for answers left without identifiers. `answer_correct` is 1 or 0 for a scheme with an `answer_field`, and `answer_reason` gives the grader's reason, such as "a different structure (ignoring stereo)" or "missing the substructure C(=O)[OH]". RDKit's `SDMolSupplier` and other SD readers open the SD file with the fields as properties.

Potato's agreement metrics do not cover `chemical_structure` or `viewer_selection` schemes, and the `verdict` radio question in the first config gets agreement as usual. To compare annotators' structures, match rows of `chemical_structures.csv` on `instance_id` and `schema` and compare `inchikey`, or `idcode` when no RDKit was available. For atom selections, compute set overlap per item from `molecule_atoms.csv`, for example Jaccard over the `atom` indices.

## Example

```bash
python potato/flask_server.py start examples/advanced/chemical-structure-correction/config.yaml -p 8000
```

The example has five structure images of common drugs and a recognized SMILES string for each image. Four of the recognized strings were edited by hand to contain a typical recognition error, such as a wrong heteroatom or an inverted stereocenter. `generate_images.py` in the same directory draws the images from the true SMILES.

## Setting up a study with a coding agent

[potato-skill](/docs/getting-started/quick-start-with-a-coding-agent) 1.2.0 targets Potato 2.13.1 and knows `chemical_structure` and `molecule` by name, and the skill guides the study-design side, such as what to ask annotators, how many annotators check each structure, training and attention checks, and export. The skill does not yet document the structure data formats, the answer-key modes, the display options, or the `chemistry` extra. Install potato-skill, describe the study to your agent, and point the agent at the Markdown copy of this page, `https://www.potatoannotator.com/docs/annotation-types/chemical-structures.md`, for the formats and options.

## Further Reading

- [Protein Structures](/docs/annotation-types/protein-structures) - Residue selection with the same `viewer_selection` scheme
- [Instance Display](/docs/core-concepts/instance-display) - How display fields are configured
- [Quality Control](/docs/features/quality-control) - Gold-standard items and attention checks
- [Export Formats](/docs/features/export-formats) - Potato's export formats and the export command
- [Life Sciences](/use-cases/life-sciences) - Potato for life sciences and biocuration

For implementation details, see the [source documentation](https://github.com/davidjurgens/potato/blob/master/docs/annotation-types/science/chemical_structures.md).