Spectrum Peak Picking and Integration

New in v2.13.0

The spectrum display draws one spectrum per item in the browser, and the peak_picking scheme records the peaks an annotator picks, the regions the annotator integrates, and the suggested peaks the annotator accepts or rejects. Chemists and spectroscopists use the pair to build labeled spectra datasets, such as peak lists for NMR spectroscopy, band assignment in infrared spectroscopy, and peak annotation in mass spectra. Instructors collect each student's peak list for the same spectrum and grade the lists after export. Groups that run an automatic peak picker load the picker's peaks as suggestions, and annotators accept or reject each suggested peak.

Peak picking lists the position and intensity of each peak in a spectrum. In a 1H NMR spectrum, the integral of a multiplet is proportional to the number of protons that give the multiplet. Annotators integrate a multiplet by dragging across the multiplet.

JCAMP-DX is a text format for spectroscopic data that IUPAC maintains. mzML is the XML format of the HUPO Proteomics Standards Initiative for mass spectrometry data. One mzML file holds the spectra of one run.

• Engine: Potato's repository includes Plotly 2.27.0 (MIT), and Potato loads Plotly only on pages with a spectrum. Plotly draws SVG, and the display needs no WebGL.
• Formats: JCAMP-DX (.jdx, .dx, .jcamp), mzML (.mzML), and two-column text (.csv, .tsv, .txt, .xy). A spectrum can also be written into the item as two lists.
• Long spectra: files are parsed on the server, and the browser receives at most max_points points (4,000 by default). Each bucket of the original spectrum keeps its lowest and highest point. A narrow NMR line therefore keeps its full height. Zooming in fetches the original points for the visible range, and the server downsamples the range again only when the range holds more than max_points.
• Axis conventions: chemical shift and wavenumber axes run from high to low, left to right. An NMR spectrum stored in Hz is converted to ppm with the file's .OBSERVE FREQUENCY.
• Offline: spectrum files are read from the media directory on the Potato server. A spectrum named by URL is refused, and Potato downloads no spectra.

JCAMP-DX and CSV files need no extra install, and peak detection needs nothing extra because scipy is a core dependency. Only mzML needs the spectra extra (pyteomics, psims, and lxml):

pip install 'potato-annotation[spectra]'

Configuration

The config below sets up 1H NMR peak picking. The display suggests peaks found by automatic detection, and annotators accept or reject each suggestion, give each peak an assignment and a multiplicity, and integrate each multiplet:

annotation_task_name: "1H NMR peak picking"
task_dir: .
media_directory: media
output_annotation_dir: annotation_output
data_files: [data/spectra.json]
item_properties: {id_key: id, text_key: note}
 
instance_display:
  fields:
    - key: spectrum               # item field naming the spectrum file
      type: spectrum
      label: "1H NMR"
      display_options:
        prepick:                  # suggested peaks, off unless set
          method: detect
          prominence: 0.05
          max_peaks: 30
 
user_config:
  allow_all_users: true
 
annotation_schemes:
  - annotation_type: peak_picking
    name: peaks
    description: "Pick the peaks, assign each one, and integrate each multiplet"
    viewer_field: spectrum
    peak_fields:
      - name: assignment
        placeholder: "e.g. OCH2"
      - name: multiplicity
        options: [s, d, t, q, m, br]
    region_fields:
      - name: protons
        label: "nH"
    min_peaks: 1
    require_review: true          # every suggestion accepted or rejected
    label_requirement: {required: true}

A field with options is a choice field, and a field without options is a text box. The display turns Integrate mode on by itself for an NMR line spectrum. With min_peaks, require_review, and label_requirement together, Next stays blocked until the annotator has picked at least one peak and has accepted or rejected every suggestion. The viewer_field of the scheme must equal the key of a spectrum field, and the server refuses to start on a mismatch.

Each item names a spectrum file relative to media_directory:

{"id": "ethyl_acetate", "note": "1H NMR, 400 MHz, CDCl3.", "spectrum": "spectra/ethyl_acetate_1h.jdx"}

Reviewing an automatic peak picker

To review a peak picker's output, put the picker's peaks in each item. Annotators can then only accept or reject each suggested peak:

annotation_task_name: "Peak picker review"
task_dir: .
media_directory: media
output_annotation_dir: annotation_output
data_files: [data/review.json]
item_properties: {id_key: id, text_key: note}
 
instance_display:
  fields:
    - key: spectrum
      type: spectrum
      display_options:
        prepick: {method: item}   # the peaks listed in each item
 
user_config:
  allow_all_users: true
 
annotation_schemes:
  - annotation_type: peak_picking
    name: review
    description: "Accept the real peaks and reject the rest"
    viewer_field: spectrum
    peak_fields: []
    region_fields: []
    require_review: true
    allow_manual: false
    label_requirement: {required: true}

{"id": "ethanol", "note": "Peaks from the automatic picker.",
 "spectrum": {"file": "spectra/ethanol_1h.jdx", "peaks": [3.69, 2.61, 1.22, {"x": 7.26, "label": "CHCl3"}]}}

A suggested peak is a number or an {x, y, label} mapping in the item's peaks list. A missing y is read from the spectrum at x, and a label is saved as the peak's label field when the annotator accepts the suggestion. With allow_manual: false, a click away from a suggestion picks nothing, and the "Add a peak at" box is removed. With peak_fields: [] and region_fields: [], the scheme asks for no text beside each peak.

Mass spectra and IR spectra

A centroided mzML scan is drawn as sticks, and a click picks the nearest stick. A JCAMP-DX file with only a peak table is drawn as sticks too, and most JCAMP-DX mass spectra are such files. The partial block below shows the display options for a mass spectrum, with the file's own peak table offered as the 20 tallest suggestions:

instance_display:
  fields:
    - key: spectrum
      type: spectrum
      display_options:
        prepick: {method: file, max_peaks: 20}

An mzML item names its scan, and a JCAMP-DX compound file names its block:

{"id": "r7", "spectrum": {"file": "runs/sample1.mzML", "scan": 1043}}
{"id": "c2", "spectrum": {"file": "spectra/compound.jdx", "block": 1}}

For an IR spectrum in transmittance or reflectance, the display picks minima. A noisy IR spectrum needs a higher prominence, such as 0.1.

Spectrum data

An item field holds a file name in the media directory, or a mapping with the keys in the table below:

Any other key is refused with a message naming the accepted keys. An inline NMR spectrum looks like the line below:

{"id": "s1", "spectrum": {"x": [0.5, 1.0, 1.5], "y": [3, 40, 2], "x_units": "PPM", "kind": "nmr"}}

Potato reads JCAMP-DX itself, including ##XYDATA in every ASDF compression form (with the DIF Y-check verified), ##XYPOINTS, ##PEAK TABLE, the ##NTUPLES form that Bruker and MestReNova write, and compound files with ##BLOCKS. A peak table next to XYDATA is the file's own peak list, and prepick method file offers the list as suggestions. FIDs without an abscissa, 2D NMR, and JCAMP-CS structures are not read.

A CSV file has two columns, x then y, separated by commas, tabs, semicolons, or spaces. Lines starting with # are comments, and an optional header row sets the units from the text in parentheses, such as shift (ppm), wavenumber (cm-1), or transmittance (%T). An mzML file that declares a DOCTYPE or entities is refused before parsing because mzML uses neither a DOCTYPE nor entities.

Options

Detection runs scipy.signal.find_peaks with the prominence threshold on the full-resolution spectrum. Suggestions are off by default because suggested peaks can anchor annotators. A study that measures how often annotators accept a picker's peaks needs suggestions on.

Picking and integrating peaks

The toolbar above the plot has three modes:

A double-click on the plot or the Reset view button zooms back out. On a touch screen, a tap works as a click.

With snap on, a click near a line picks the line's top, and annotators zoom in to separate the lines of a multiplet before picking each line. Suggested peaks are drawn as open orange triangles, and picked peaks as filled triangles labeled with the peak's field values. Each region is shaded and labeled with the region's integral relative to the first region, and the first region reads 1.00.

The server computes each integral as a trapezoid integral over every original point between the region's ends, with no baseline correction. A region drawn wider than its multiplet therefore reads high. When the relative integrals need a fixed reference, say in the instructions which signal to integrate first. For a stick spectrum with integrate: true, the integral is the summed intensity of the sticks in the region.

Under the plot, the scheme lists the suggested peaks with Accept and Reject buttons, plus Accept all and Reject all. A rejected suggestion appears as a small button, and pressing the button restores the suggestion. The Peaks table has one row per peak, with the peak's position, intensity, source, and fields, and a Remove button. The Integrals table has one row per region, with the region's range, integral, relative integral, and fields. With the "Add a peak at" box, an annotator types a position and presses Enter, and a position that matches a suggestion accepts the suggestion. Every control is a button or a form field. An annotator can therefore review a spectrum from the keyboard alone.

Stored answer

A peak_picking scheme stores one JSON object per item:

{"viewer": "spectrum",
 "items": [{"kind": "peak", "x": 2.0399, "y": 3003, "source": "auto", "width": 0.0031,
            "fields": {"assignment": "CH3CO", "multiplicity": "s"}},
           {"kind": "peak", "x": 7.26, "y": 132, "source": "manual"},
           {"kind": "region", "from": 1.969, "to": 2.120, "integral": 15.77,
            "fields": {"protons": "3"}}],
 "rejected": [4.1464]}

source is auto for an accepted suggestion and manual for a peak the annotator added. width is the full width at half prominence of a detected peak, in x units. rejected lists the positions of the suggestions the annotator rejected. A suggestion the annotator neither accepted nor rejected is not stored, and require_review in a required scheme makes sure every suggestion ends up in one of the two lists.

Export

The peak_table export writes one row per peak, region, and rejected suggestion:

python -m potato.export --config config.yaml --format peak_table --output out/

The export writes peaks.csv and peaks.jsonl, and --option format=csv or --option format=jsonl writes only the matching file. The JSONL rows leave out empty columns.

Several annotators can pick peaks on the same spectrum, and num_annotators_per_item sets how many. The generic CSV and JSONL exports also hold the answer, as one JSON cell per scheme. Potato's agreement metrics do not cover peak_picking schemes. Compare annotators' peak lists from peaks.csv after export (e.g., by matching peaks of one item within a tolerance in ppm or cm⁻¹). For a peak picker review, the counts of source: auto peak rows and rejected rows give each annotator's acceptance rate.

Example

python potato/flask_server.py start examples/advanced/spectrum-peak-picking/config.yaml -p 8000

The example has two synthetic 1H NMR spectra (ethyl acetate and ethanol) as DIFDUP-compressed JCAMP-DX in Hz and one synthetic IR transmittance spectrum as CSV. Detected suggestions and require_review are on. All three spectra are invented. generate_spectra.py in the same directory regenerates the files.

Setting up a study with a coding agent

potato-skill 1.2.0 targets Potato 2.13.1 and knows peak_picking and spectrum by name. The skill guides the study-design side, including what to ask annotators, how many annotators pick peaks on each spectrum, training and attention checks, and export. The skill does not yet document the spectrum file formats, the display options, or the spectra extra. Install potato-skill, describe the study to your agent, and point the agent at the Markdown copy of this page for the formats and options. The Markdown copy is at https://www.potatoannotator.com/docs/annotation-types/spectra.md.

Further reading

• Chemical Structures - Structure editing and atom selection on molecules
• Instance Display - How display fields are configured
• Export Formats - Potato's export formats and the export command
• How Many Annotators Do You Need? - Choosing annotators per item
• Life Sciences - Potato for life sciences and biocuration

For implementation details, see the source documentation.