Chemical Structure Annotation and OCSR Correction

New in v2.13.0

The chemical_structure annotation scheme gives annotators a structure editor, and the molecule display draws a small molecule on which annotators select atoms. Chemistry dataset builders use the pair for structure correction. In structure correction, annotators compare a structure recognized from a figure in a patent or paper with the figure and fix the misread atoms and bonds. Chemistry education researchers and instructors use the editor to collect drawn structures, such as the structure a compound name describes, and Potato grades each drawing against an answer key on the server. Curators use the molecule display with a viewer_selection scheme to mark atoms, such as a reaction center or a site of metabolism. Each answer is stored as a molfile with the drawing's canonical SMILES, and the chemistry export writes every annotator's structures to a CSV file and an SD file.

Optical chemical structure recognition (OCSR) converts an image of a chemical structure into a machine-readable structure. Potato runs no OCSR model, and a study loads the recognized structures produced by another tool. SMILES writes a structure as a short line of text, such as CC(=O)O for acetic acid. An MDL molfile is the connection-table text format of the chemical table file family, and an SD file holds many molfiles with named data fields after each one. The InChIKey is a 27-character hash of the IUPAC InChI identifier, and two drawings of the same structure give the same InChIKey.

• Editor: OpenChemLib JS 9.25.0 (BSD-3-Clause), committed with Potato. The editor draws on a canvas with a toolbar for bonds, rings, elements, charges, stereo wedges, erasing, and undo. A text box under the canvas takes a typed or pasted SMILES string or molfile, and the text box is the keyboard route into the editor.
• RDKit.js (optional): the RDKit JavaScript MinimalLib 2026.3.6 (BSD-3-Clause) is a 7 MB WebAssembly build and is not committed. potato vendor fetch rdkit downloads the build and checks the download against the npm registry's integrity hash. With RDKit.js, the browser also computes the RDKit canonical SMILES, the InChI, and the InChIKey, flags structures RDKit cannot sanitize, and the molecule display can draw with engine: rdkit.
• The chemistry extra (optional): installs the rdkit Python package on the server. The package grades answer keys, and the chemistry export uses the package to fill in InChI and InChIKey for answers saved without RDKit.js.
• No WebGL: both toolkits draw in 2D.
• Offline: structures come from the item or from a file in the media directory, and answer keys never reach the browser.

The features available with each install are:

Server-side grading needs the extra, and RDKit.js is a separate fetch:

pip install 'potato-annotation[chemistry]'
potato vendor fetch rdkit     # optional, InChIKeys in the browser

Restart the server after the fetch. While RDKit.js is missing, potato validate warns for each chemical_structure scheme left at rdkit: auto and names the fetch command.

Configuration

The config below sets up OCSR correction. Each item shows the source figure beside the recognized structure, and annotators mark the atoms the recognizer misread, correct the structure in the editor, and give a verdict:

annotation_task_name: "Structure recognition check"
task_dir: .
media_directory: media
output_annotation_dir: annotation_output
data_files: [data/structures.json]
item_properties: {id_key: id, text_key: note}
 
instance_display:
  layout:
    direction: horizontal
  fields:
    - key: page_image             # the figure cropped from the patent or paper
      type: image
      label: "Source image"
    - key: ocsr_smiles            # what the OCSR tool produced
      type: molecule
      label: "Recognized structure"
      display_options:
        height: 300
 
user_config:
  allow_all_users: true
 
annotation_schemes:
  - annotation_type: viewer_selection
    name: wrong_atoms
    description: "Mark the atoms the recognizer got wrong"
    viewer_field: ocsr_smiles
    labels: ["wrong element", "wrong bond or charge", "wrong stereo"]
 
  - annotation_type: chemical_structure
    name: corrected
    description: "Correct the structure so it matches the source image"
    prefill_field: ocsr_smiles
    answer_field: true_smiles     # graded in the export, never sent to the page
    label_requirement: {required: true}
 
  - annotation_type: radio
    name: verdict
    description: "How good was the recognition?"
    labels: ["Correct", "Minor errors", "Wrong structure"]
    label_requirement: {required: true}

The editor starts from ocsr_smiles. A pre-filled structure that the annotator leaves as given becomes the saved answer with changed: false, and a correct recognition therefore needs no edit. The verdict question separates an accepted structure from an item the annotator never checked. Potato removes the true_smiles field from everything the page receives. In the export, answer_correct says whether each corrected structure matches true_smiles. Comparing answer_correct with changed separates missed errors (unchanged and wrong) from introduced errors (changed and wrong).

Each item names the figure relative to media_directory and gives both structures:

{"id": "fig1_aspirin", "note": "Compound 1 as printed.", "page_image": "fig1_aspirin.png",
 "ocsr_smiles": "CC(=O)Oc1ccccc1C(=O)N", "true_smiles": "CC(=O)Oc1ccccc1C(=O)O"}

Drawing a structure from a name

A name-to-structure exercise needs one scheme and an answer key per item. The config below shows the compound name as the item text:

annotation_task_name: "Name to structure"
task_dir: .
output_annotation_dir: annotation_output
data_files: [data/names.json]
item_properties: {id_key: id, text_key: name}
 
user_config:
  allow_all_users: true
 
annotation_schemes:
  - annotation_type: chemical_structure
    name: drawn
    description: "Draw the structure this name describes"
    answer_field: expected
    rdkit: false                  # OpenChemLib only, no RDKit.js warning
    editor_height: 420
    label_requirement: {required: true}

{"id": "q2", "name": "(S)-2-aminopropanoic acid", "expected": "C[C@H](N)C(=O)O"}

Students can draw with the mouse or type a SMILES string and press Enter. With rdkit: false, the page loads OpenChemLib only and stores no InChIKey, and the chemistry extra grades the drawings at export.

Marking atoms on a molecule

Atom marking pairs a molecule display with a viewer_selection scheme. The config below asks curators to mark sites of metabolism and draws a model's predicted sites in amber for reference:

annotation_task_name: "Sites of metabolism"
task_dir: .
output_annotation_dir: annotation_output
data_files: [data/molecules.json]
item_properties: {id_key: id, text_key: note}
 
instance_display:
  fields:
    - key: smiles
      type: molecule
      display_options:
        show_atom_numbers: true
        highlight_field: predicted_site   # a model's predicted atoms, drawn in amber
 
user_config:
  allow_all_users: true
 
annotation_schemes:
  - annotation_type: viewer_selection
    name: som
    description: "Mark each atom where the compound is metabolized"
    viewer_field: smiles
    labels: ["hydroxylation", "N-dealkylation", "glucuronidation"]
    min_selections: 1
    label_requirement: {required: true}

{"id": "m1", "note": "Predicted sites in amber.", "smiles": "CC(=O)Nc1ccc(O)cc1", "predicted_site": [0, 8]}

The item field that highlight_field names holds 0-based atom indices, as a list or as text such as "0, 8". The highlight is fixed, and curators select atoms independently of the highlight. With min_selections and label_requirement together, Next stays blocked until the curator has selected at least one atom.

Structure data

A structure field holds a SMILES string, molfile text, or the name of a .mol, .sdf, .smi, or .smiles file in the media directory. The server reads the first record of an .sdf file and the first token of the first non-empty line of a .smi file. Structure text and structure files are limited to 1 MB. A value with spaces that is not a molfile is refused, because a SMILES string has no spaces. The display never fetches a URL, and Potato refuses a file path outside the media directory.

Options

Answer keys

An answer key is a bare SMILES string, a molfile, or a mapping. All parts of a mapping must pass:

The exact mode compares standard InChIKeys. Standard InChI treats some tautomers as one structure, and a drawing of 2-pyridone therefore passes against a key of 2-hydroxypyridine. Charges count, and ethoxide fails against ethanol even with ignore_stereo. A substructure key grades a feature of the answer, such as {"substructure": "C(=O)[OH]"} for a carboxylic acid.

An answer_field value may be a bare SMILES string. Gold-standard items and training items write the mapping form under the scheme's name:

{"id": "gold_1", "gold_label": {"corrected": {"smiles": "CC(=O)Oc1ccccc1C(=O)O"}}}
{"id": "train_1", "correct_answers": {"drawn": {"substructure": "C(=O)[OH]"}}}

Quality control and the training phase grade these keys with the same comparison as the export. The editor reads prefill_field only in the annotation phase, and a training item therefore opens with an empty canvas.

Without the rdkit Python package, a smiles, molfile, or SMARTS key grades every answer wrong, and the server logs the missing package once. One case still works. An inchikey key matches an answer whose InChIKey RDKit.js computed in the annotator's browser. With the package installed, the server recomputes the InChIKey from the stored molfile and ignores the browser's value.

Drawing and correcting structures

To add a bond, the annotator clicks an empty part of the canvas or drags from an atom, after choosing a bond type, ring, element, or charge in the toolbar on the left. Pointing at an atom and typing an element symbol (o, n, s) changes the element. The text box takes a SMILES string or molfile, and Load (or Enter, for a SMILES string) replaces the drawing. Text OpenChemLib cannot read is refused, and the drawing is kept. Clear empties the canvas, and Restore the given structure brings back the pre-filled structure.

The SMILES, formula, and InChIKey under the editor update while the annotator draws. Valence problems are listed there in the form "C (atom 2) has 5 bonds; it allows 4.", and with require_valid the problems keep a required item from being submitted.

Selecting atoms

Annotators click an atom in the drawing to select the atom, and click again to remove the selection. In the atom list, Tab moves to the list, the arrow keys, Home, and End move between atoms, and Space or Enter toggles an atom. With labels, the annotator picks a label first, and each listed atom has a label menu and a Remove button. The page names atoms by element and molfile number, and O4 is the fourth atom of the source.

A stored atom index is the 0-based position of the atom in the source as written, in the order of atoms in the SMILES string or the molfile's atom block. The molfile atom number is the index plus 1. The index equals RDKit's atom index for the same input read with removeHs=False, whichever engine draws the molecule. OpenChemLib folds explicit hydrogens in a SMILES string ([H]C(Cl)(Br)F) into the neighboring atom, and with engine: openchemlib such a SMILES string cannot be used for selection. The display says so and offers no atoms at all. Use a molfile or engine: rdkit when individual hydrogens must be selectable. Implicit hydrogens (CCO) are never selectable.

Stored answer

A chemical_structure scheme stores one JSON object per item:

{"molfile": "\nOCL MolfileCreator  2D\n\n  0  0  0 ... V3000\n...M  END\n",
 "smiles": "CC(Oc1ccccc1C(O)=O)=O",
 "idcode": "dklB@@QmR[fUxUZBBF@@",
 "formula": "C9H8O4",
 "rdkit_smiles": "CC(=O)Oc1ccccc1C(=O)O",
 "inchi": "InChI=1S/C9H8O4/c1-6(10)13-8-5-3-2-4-7(8)9(11)12/h2-5H,1H3,(H,11,12)",
 "inchikey": "BSYNRYMUTXBXSQ-UHFFFAOYSA-N",
 "changed": true,
 "problems": [],
 "toolkits": {"openchemlib": "9.25.0", "rdkit": "2026.03.6"}}

The molfile is V3000. smiles is OpenChemLib's canonical isomeric SMILES, and idcode is OpenChemLib's canonical ID code. rdkit_smiles, inchi, and inchikey are present only when RDKit.js ran. Canonical SMILES differ between toolkits, and structures should therefore be compared by InChIKey or ID code, never by smiles against rdkit_smiles. changed is false while the drawing has the pre-filled structure's ID code.

A viewer_selection scheme on a molecule display stores the selected atoms:

{"viewer": "molecule", "items": [{"atom": 12, "element": "N", "label": "wrong element"}]}

Export

The chemistry export covers chemical_structure schemes and viewer_selection schemes bound to a molecule display:

python -m potato.export --config config.yaml --format chemistry --output out/

identifiers_from is browser when RDKit.js computed the InChI and InChIKey, server when the export computed both with the rdkit Python package, and none when neither was available. The export lists a warning for answers left without identifiers. answer_correct is 1 or 0 for a scheme with an answer_field, and answer_reason gives the grader's reason, such as "a different structure (ignoring stereo)" or "missing the substructure C(=O)[OH]". RDKit's SDMolSupplier and other SD readers open the SD file with the fields as properties.

Potato's agreement metrics do not cover chemical_structure or viewer_selection schemes, and the verdict radio question in the first config gets agreement as usual. To compare annotators' structures, match rows of chemical_structures.csv on instance_id and schema and compare inchikey, or idcode when no RDKit was available. For atom selections, compute set overlap per item from molecule_atoms.csv, for example Jaccard over the atom indices.

Example

python potato/flask_server.py start examples/advanced/chemical-structure-correction/config.yaml -p 8000

The example has five structure images of common drugs and a recognized SMILES string for each image. Four of the recognized strings were edited by hand to contain a typical recognition error, such as a wrong heteroatom or an inverted stereocenter. generate_images.py in the same directory draws the images from the true SMILES.

Setting up a study with a coding agent

potato-skill 1.2.0 targets Potato 2.13.1 and knows chemical_structure and molecule by name, and the skill guides the study-design side, such as what to ask annotators, how many annotators check each structure, training and attention checks, and export. The skill does not yet document the structure data formats, the answer-key modes, the display options, or the chemistry extra. Install potato-skill, describe the study to your agent, and point the agent at the Markdown copy of this page, https://www.potatoannotator.com/docs/annotation-types/chemical-structures.md, for the formats and options.

Further Reading

• Protein Structures - Residue selection with the same viewer_selection scheme
• Instance Display - How display fields are configured
• Quality Control - Gold-standard items and attention checks
• Export Formats - Potato's export formats and the export command
• Life Sciences - Potato for life sciences and biocuration

For implementation details, see the source documentation.