Keller and Vosshall Odor Ratings from Molecular Structure

# Odor perception rating for chemically diverse molecules (Keller and Vosshall 2016)
#
# Dataset: Keller and Vosshall (2016). 55 analyzed subjects rated the smell of
# 480 molecules at two dilutions. Each subject sniffed a physical sample and
# answered questions on a computer. The ratings are public in the paper's
# additional file, and the Pyrfume copy (keller_2016) used for the reference
# fields here re-organizes that file.
#
# THIS IS A LARGE ADAPTATION. The original raters smelled a vial. Annotators
# here see only the molecule's structure and predict the percept, so the design
# is a structure-to-perception rating task and a review of the published
# ratings, not a reproduction of smelling. The numbers an annotator enters are
# guesses about a smell, never measurements of one.
#
# Paper:  Keller and Vosshall, Olfactory perception of chemically diverse
#         molecules. BMC Neuroscience 17, 2016.
# DOI:    10.1186/s12868-016-0287-2
# BibTeX (Crossref):
#   @article{Keller_2016,
#     title   = {Olfactory perception of chemically diverse molecules},
#     volume  = {17},
#     ISSN    = {1471-2202},
#     url     = {http://dx.doi.org/10.1186/s12868-016-0287-2},
#     DOI     = {10.1186/s12868-016-0287-2},
#     number  = {1},
#     journal = {BMC Neuroscience},
#     publisher = {Springer Science and Business Media LLC},
#     author  = {Keller, Andreas and Vosshall, Leslie B.},
#     year    = {2016},
#     month   = Aug
#   }
# Data:   https://github.com/pyrfume/pyrfume-data/tree/main/keller_2016
#         (molecules.csv, stimuli.csv, behavior.csv). Licence: the article is
#         CC BY 4.0 and "The Creative Commons Public Domain Dedication waiver
#         (http://creativecommons.org/publicdomain/zero/1.0/) applies to the
#         data made available in this article, unless otherwise stated."
#         The ten molecules and their reference means in sample-data.json are
#         real values taken from that data. The reference fields are listed in
#         item_properties.private_keys, so annotators do not see them.
#
# Protocol quoted from the paper (Methods, "General psychophysics procedures").
#   "Subjects opened the vial, sniffed the contents, and were asked if they
#   could smell anything. If the answer was "no," they were directed to move on
#   to the next stimulus."
#   "The first three questions asked how familiar, strong, and pleasant the
#   smell was. For these three questions, the slider started in the middle of
#   the line (position 50) and subjects were required to move it. The other 20
#   questions were how well each of 20 descriptors ... applied to the smell.
#   For these questions, the slider started at the bottom of the line (position
#   0) and subjects were not required to move it."
#   "The final position of the slider was then translated into a scale from 0
#   to 100."
#   The 20 descriptors were "edible," "bakery," "sweet," "fruit," "fish,"
#   "garlic," "spices," "cold," "sour," "burnt," "acid," "warm," "musky,"
#   "sweaty," "ammonia/urinous," "decayed," "wood," "grass," "flower," and
#   "chemical."
# Question and answer wording in the dataset (behavior.csv): "CAN OR CAN'T
# SMELL" with the answers "I smell something" and "I can't smell anything";
# "HOW STRONG IS THE SMELL?", "HOW PLEASANT IS THE SMELL?", "HOW FAMILIAR IS
# THE SMELL?". The paper does not describe the captions at the two ends of the
# slider, so the design states only the 0 to 100 range.
#
# Adaptations (this design differs from the original protocol here):
#   - The wording of the five descriptor questions ("How well does the
#     descriptor ... apply to the smell?") paraphrases the paper's Methods
#     sentence. The dataset stores only the bare descriptor names. The wording
#     of the can-smell, strength, pleasantness, and familiarity questions
#     follows the dataset's prompts.
#   - Structure only. Annotators see a drawing of the molecule (SMILES) and
#     predict how a person would rate its smell, instead of smelling a vial.
#   - Five of the 20 descriptors (sweet, fruit, garlic, burnt, chemical), to
#     keep the form short. The other 15 are left out.
#   - Left out of the original sequence: "know what the smell is" and the
#     optional description in the subject's own words.
#   - Ten molecules at the 1/1000 dilution. The experiment used 480 molecules
#     at two dilutions (1,000 stimuli including 20 repeats).
#   - The molecule's name and the published means (reference_* fields in
#     sample-data.json) are in the data and not shown on the page. They are for
#     comparing annotators' guesses with the published ratings afterward.
#   - "Can smell" is answered about a typical subject, not about yourself.
#     Published ratings exist only for subjects who answered "I smell
#     something", so the reference means average those subjects.
#
# Training (model_training). The block turns training on and you choose the
# rdkit-property trainer on /admin/training. That trainer reads the
# molecule display field, turns each structure into a Morgan fingerprint plus
# RDKit 2D descriptors, and fits a scikit-learn model on one scheme at a time
# (logistic regression or a random forest for the radio, ridge or a random
# forest for a slider, rounded to the slider's step). It predicts from
# structure to the annotators' answers, so it learns the annotators' guesses,
# not odor. Use the admin key below. It needs
# `pip install "potato-annotation[chemistry]"` and, by default, at least ten
# labelled molecules in its training split, so add more molecules before
# training. It does not serve or deploy a model.

annotation_task_name: "Odor perception rating from molecular structure"
task_dir: "."
data_files:
  - sample-data.json

item_properties:
  id_key: id
  text_key: smiles
  private_keys:
    - molecule_name
    - reference_n_could_smell
    - reference_n_raters
    - reference_mean_intensity
    - reference_mean_pleasantness
    - reference_mean_familiarity

output_annotation_dir: annotation_output
export_annotation_format: jsonl

require_password: false
admin_api_key: "demo-admin-key"

instance_display:
  fields:
    - key: smiles
      type: molecule
      label: "Molecule"
      display_options:
        engine: openchemlib
        height: 300
        show_atom_numbers: false

user_config:
  allow_all_users: true

annotation_schemes:
  - annotation_type: radio
    name: can_smell
    description: "CAN OR CAN'T SMELL. A person sniffs a vial of this molecule diluted 1/1000. Would they report that they smell something?"
    labels:
      - "I smell something"
      - "I can't smell anything"
    label_requirement:
      required: true

  - annotation_type: slider
    name: intensity
    description: "HOW STRONG IS THE SMELL? Rate from 0 to 100. The original slider started at 50 and the subject had to move it. Skip this if you answered that nothing can be smelled."
    min_value: 0
    max_value: 100
    starting_value: 50
    step: 1

  - annotation_type: slider
    name: pleasantness
    description: "HOW PLEASANT IS THE SMELL? Rate from 0 to 100. The original slider started at 50 and the subject had to move it."
    min_value: 0
    max_value: 100
    starting_value: 50
    step: 1

  - annotation_type: slider
    name: familiarity
    description: "HOW FAMILIAR IS THE SMELL? Rate from 0 to 100. The original slider started at 50 and the subject had to move it."
    min_value: 0
    max_value: 100
    starting_value: 50
    step: 1

  - annotation_type: slider
    name: descriptor_sweet
    description: "How well does the descriptor \"sweet\" apply to the smell? Rate from 0 to 100. The original slider started at 0 and was optional."
    min_value: 0
    max_value: 100
    starting_value: 0
    step: 1

  - annotation_type: slider
    name: descriptor_fruit
    description: "How well does the descriptor \"fruit\" apply to the smell? Rate from 0 to 100. The original slider started at 0 and was optional."
    min_value: 0
    max_value: 100
    starting_value: 0
    step: 1

  - annotation_type: slider
    name: descriptor_garlic
    description: "How well does the descriptor \"garlic\" apply to the smell? Rate from 0 to 100. The original slider started at 0 and was optional."
    min_value: 0
    max_value: 100
    starting_value: 0
    step: 1

  - annotation_type: slider
    name: descriptor_burnt
    description: "How well does the descriptor \"burnt\" apply to the smell? Rate from 0 to 100. The original slider started at 0 and was optional."
    min_value: 0
    max_value: 100
    starting_value: 0
    step: 1

  - annotation_type: slider
    name: descriptor_chemical
    description: "How well does the descriptor \"chemical\" apply to the smell? Rate from 0 to 100. The original slider started at 0 and was optional."
    min_value: 0
    max_value: 100
    starting_value: 0
    step: 1

model_training:
  enabled: true
  writeback:
    schemas: ["pleasantness", "intensity", "can_smell"]
    include_unlabeled: true
    min_confidence: 0.6

annotation_instructions: |
  This task is a structure-to-perception rating based on Keller and Vosshall (2016). You cannot smell the sample, so rate how you expect an average person to perceive it from the molecule's structure alone. First say whether the molecule would be smelled at all, then rate strength, pleasantness, familiarity, and five of the paper's twenty descriptors on 0 to 100 sliders. Your answers are predictions, not measurements.