Real-Bogus Vetting and Galaxy Morphology on FITS Cutouts

New in v2.13.0

The fits_cutout display shows FITS images in the browser, either one image per item or a transient candidate's science, reference, and difference cutouts side by side above the candidate's light curve. Time-domain astronomers use the display for real-bogus vetting. In real-bogus vetting, annotators label each candidate from a survey such as ZTF or Rubin as a real astrophysical transient or a bogus detection. The labels become training sets for real-bogus classifiers. For example, the Braai classifier for ZTF was trained on more than 30,000 labeled science, reference, and difference triplets (Duev et al., MNRAS 489, 3582). Galaxy morphology classification uses one image per item and a Galaxy-Zoo-style question tree. The display itself stores nothing. The vetting question is usually an ordinary radio scheme with Real, Bogus, and Unsure on the R, B, and U keys.

FITS is the standard file format for astronomical images and tables. The World Coordinate System (WCS) keywords in a FITS header map each pixel of an image to a position on the sky. A difference image is a calibrated science exposure minus a reference (template) image of the same field, and a real transient appears in the difference image as a new point source. ZTF alert packets hold the science, reference, and difference cutouts as gzip-compressed FITS images (alert schema), and Rubin staff have said that Rubin alerts will hold all three cutouts as well (Rubin forum).

• Reading: astropy (BSD-3-Clause) reads the FITS files on the Potato server, cuts out the region around the item's position, scales the cutout, and sends a PNG with a description of the cutout's WCS. The server imports astropy when the first FITS file is read and starts without loading astropy.
• Formats: .fits, .fit, .fts, and gzipped .fits.gz. The display shows the HDU named with hdu, or else the first HDU holding a 2-D (or larger) image. For a cube, the display shows the cube's first plane. A ZTF cutout written to disk with a .fits.gz name opens without decompression.
• Cutouts: the center is a position in FITS pixels (x, y) or on the sky (ra, dec), and the size is pixels or an angle such as "40 arcsec". Only the rows of the cutout are read from an uncompressed file.
• Caching: each converted cutout is written to <task_dir>/.potato_cache/fits/ as a PNG and a JSON file. The cache key covers the file's path, size, modification time, and the scaling options. An unchanged file is therefore converted once. The cache keeps the 4,000 most recently used entries, and deleting the folder is safe.
• Offline: FITS files and light curves are read from the media directory. URLs and paths outside the media directory are refused, and the light curve plot uses the copy of Plotly that ships with Potato.

The display needs the astro extra (astropy>=5.3):

pip install 'potato-annotation[astro]'

A config with a fits_cutout field and no astropy installed fails at startup with the install command in the message.

Configuration

The config below sets up real-bogus vetting on cutout triplets. A real-bogus label is required for every candidate, and a second question about the cause of a bogus detection appears only after an annotator picks Bogus:

annotation_task_name: "Transient candidate vetting"
task_dir: .
media_directory: media
output_annotation_dir: annotation_output
data_files: [data/candidates.json]
item_properties: {id_key: id, text_key: summary}
 
instance_display:
  fields:
    - key: candidate            # item field holding the triplet
      type: fits_cutout
      label: "Cutouts"
      display_options:
        size: 48                # pixels, used when an item gives no size
        interval: zscale
        stretch: linear
        panels:
          - {key: science, label: Science}
          - {key: reference, label: Reference}
          - {key: difference, label: Difference, stretch: asinh}
 
user_config:
  allow_all_users: true
 
annotation_schemes:
  - annotation_type: radio
    name: vetting
    description: "Is this a real astrophysical transient?"
    humanize_labels: false
    labels:
      - {name: "Real", key_value: "r"}
      - {name: "Bogus", key_value: "b"}
      - {name: "Unsure", key_value: "u"}
    label_requirement: {required: true}
 
  - annotation_type: multiselect
    name: bogus_reason
    description: "What makes the candidate bogus?"
    humanize_labels: false
    labels:
      - "Subtraction residual (dipole)"
      - "Cosmic ray or hot pixel"
      - "Satellite or asteroid trail"
      - "Bad column or image edge"
      - "Other"
    display_logic:
      show_when:
        - {schema: vetting, operator: equals, value: "Bogus"}

The panels list picks the cutouts and their order, and each panel can set the panel's own scaling. The difference panel here uses an asinh stretch, and the science and reference panels keep the display's zscale interval and linear stretch. humanize_labels: false shows each label as written. The "Bogus" on screen then matches the value in the display_logic condition. A hidden question is skipped by the required-answer check on the page and on the server.

Each item gives the three FITS paths relative to media_directory, the candidate's position, and an optional light curve:

{
  "id": "cand-001",
  "summary": "cand-001: ra 150.11920, dec +2.20583, band r",
  "candidate": {
    "science": "cutouts/cand-001_sci.fits",
    "reference": "cutouts/cand-001_ref.fits",
    "difference": "cutouts/cand-001_diff.fits",
    "ra": 150.11920, "dec": 2.20583,
    "size": 48,
    "light_curve": "lightcurves/cand-001.csv"
  }
}

For galaxy morphology, an item field holding one path shows one image:

{"id": "gal-17", "image": "galaxies/gal-17.fits"}

ZTF light curves

Potato finds light curve columns by common names, and ZTF's photometry columns magpsf and sigmapsf are not among the names. Without a mapping, a ZTF light curve with jd, fid, magpsf, sigmapsf, diffmaglim columns draws each row that has a diffmaglim as an upper limit. The partial block below adds two options to the fits_cutout field of the config above and names each column:

instance_display:
  fields:
    - key: candidate
      type: fits_cutout
      display_options:
        light_curve_columns: {time: jd, band: fid, mag: magpsf, err: sigmapsf, limit: diffmaglim}
        time_label: JD

The "this image" marker on the plot comes from the science image's MJD-OBS, and Potato draws the marker only when MJD-OBS lies no more than 1,000 time units before the first point or after the last point. On a JD axis, MJD-OBS is about 2,400,000 smaller than every time, and Potato draws no marker. An epoch in JD on each item places the marker. ZTF filter ids 1, 2, and 3 become band names "1", "2", and "3", and writing g, r, and i into the band column instead gives each band its usual color.

Galaxy morphology with a question tree

A Galaxy-Zoo-style decision tree asks a fixed sequence of questions, and each answer decides which question comes next. The Galaxy Zoo DECaLS tree opens with "Smooth or Featured?" (Walmsley et al., MNRAS 509, 3966). In Potato, each question of the tree is a radio scheme, and display_logic on each follow-up shows the question only after the answers that lead to the question.

Two rules of display_logic shape the tree. First, a hidden question keeps the answer an annotator gave before the question was hidden. Second, a question's visibility is computed from the answers alone, whether or not the parent question is on screen. Each condition therefore joins the whole path from the first question with all. A condition that names only the parent leaves a later question on screen after an annotator changes an early answer.

The config below asks a four-question tree on single galaxy images:

annotation_task_name: "Galaxy morphology"
task_dir: .
media_directory: media
output_annotation_dir: annotation_output
data_files: [data/galaxies.json]
item_properties: {id_key: id, text_key: id}
 
instance_display:
  fields:
    - key: image
      type: fits_cutout
      label: "Galaxy"
      display_options:
        interval: percentile
        percentile: 99.5
        stretch: asinh
 
user_config:
  allow_all_users: true
 
annotation_schemes:
  - annotation_type: radio
    name: shape
    description: "Is the galaxy simply smooth and rounded, with no sign of a disk?"
    humanize_labels: false
    labels: ["Smooth", "Features or a disk", "Star or artifact"]
    label_requirement: {required: true}
 
  - annotation_type: radio
    name: edge_on
    description: "Could this be a disk viewed edge-on?"
    labels: ["Yes", "No"]
    label_requirement: {required: true}
    display_logic:
      show_when:
        - {schema: shape, operator: equals, value: "Features or a disk"}
 
  - annotation_type: radio
    name: bulge_edge_on
    description: "Does the edge-on disk have a bulge at its center? If so, what shape?"
    humanize_labels: false
    labels: ["Rounded", "Boxy", "No bulge"]
    label_requirement: {required: true}
    display_logic:
      show_when:
        - all:
            - {schema: shape, operator: equals, value: "Features or a disk"}
            - {schema: edge_on, operator: equals, value: "Yes"}
 
  - annotation_type: radio
    name: spiral
    description: "Is there any sign of a spiral arm pattern?"
    labels: ["Yes", "No"]
    label_requirement: {required: true}
    display_logic:
      show_when:
        - all:
            - {schema: shape, operator: equals, value: "Features or a disk"}
            - {schema: edge_on, operator: equals, value: "No"}

With no position in the item, the display shows the whole image. A percentile interval with an asinh stretch shows a galaxy's faint outer disk and bright core in one view. The zscale interval is tuned for faint sources and can saturate a bright core. Quote "Yes" and "No" in the YAML because unquoted Yes and No are read as booleans. The upstream example puts a tree of the same kind after the vetting question. The example's tree asks about the host galaxy of each real candidate.

Item data

An item's field is a FITS path or a mapping with the keys below:

Scaling options apply at four levels, lowest first: (1) the display options, (2) the panels entry, (3) the item, and (4) the item's mapping for one image. A later level overrides an earlier level. An item with a position and no size anywhere gets a 64-pixel cutout, and a cutout is at most 4,096 pixels on a side. An item without a position shows the whole image, and an image wider or taller than 1,024 pixels is block-averaged down to 1,024. Whole images are limited to max_image_pixels. A size without a position is an error. Without a panels list, the display shows whichever of science, reference, and difference the item has, or else image.

Light curve files

A light curve file has one row per point:

mjd,band,mag,magerr,limit
60570.0,r,,,21.3
60600.0,g,20.21,0.07,
60603.0,g,19.37,0.05,

A row with a magnitude is a detection, and a row with no magnitude and a limit is an upper limit. Rows with no usable time are skipped and counted. Without light_curve_columns, column names are matched without regard to case against the lists below, in order:

A JSON light curve is a list of point objects, or an object with the list under points or data. A TSV file is detected by the .tsv extension or by the tabs in the file, and lines starting with # are skipped. Files over 2 MB are refused, and rows past 20,000 are dropped. Bands are ordered u, g, r, i, z, y, J, H, K, W1, W2, c, and o, with any other band name after the listed bands.

Display options

Options go under display_options or directly on the field, and the server checks every value at startup:

The engine option takes one value, potato. The display refuses allow_remote because FITS files are always read on the server.

Viewer controls

The panels share zoom, pan, and one crosshair, and zoom steps run from 1× to 16×:

The annotation keys (R, B, and U in the vetting config) keep working while a cutout has focus. The arrow keys move the crosshair only while a cutout has focus, and elsewhere the arrow keys move between items as usual. Blank (NaN or BLANK) pixels and any part of a cutout past the image edge are transparent and show as a checkerboard, and a status line under the panels says which applies.

The panels count as aligned when the panels have the same size, pixel scales within 1%, and centers within half a pixel on the sky. Panels that are not aligned get a note, and Potato places the crosshair on each panel by sky position with each panel's linear TAN WCS. The light curve plot marks the science image's epoch as "this image", draws upper limits as open downward triangles, and lists every point in a table under the plot.

The WCS sent to the browser

The browser never reads FITS. For each cutout, the server sends the PNG's URL and a WCS description with the cutout's CRPIX, CRVAL, and CD matrix, the pixel scale, the north and east directions, and a grid of sky positions sampled every 16 output pixels across the cutout. The readout interpolates the grid and follows any projection astropy reads (SIP distortion included). The grid stores each sky position as a unit vector, and interpolating unit vectors keeps the readout continuous across RA 0/360 and near the poles. The linear keywords leave out SIP terms, and the crosshair placement for unaligned panels uses the linear keywords. Galactic images label the axes l and b, and ecliptic images label the axes λ and β.

Stored answer and export

The display stores nothing. Each scheme on the page stores the annotator's answer as for any radio or multiselect question, and the answers export with the generic formats:

python -m potato.export -c config.yaml -f csv -o exports/csv

The CSV has one row per annotator and item, with a column per scheme and label, such as vetting.Real and bogus_reason.Cosmic ray or hot pixel. -f jsonl writes the same records as JSON lines. An answer to a tree question that was later hidden stays in the stored annotation. Filter follow-up answers by the path of earlier answers before analysis.

Potato's agreement metrics cover these schemes. A radio question such as vetting gets percent agreement, Cohen's and Fleiss' kappa, and Krippendorff's alpha, and a multiselect question such as bogus_reason gets mean Jaccard and Krippendorff's alpha with MASI distance. The consensus export added in 2.13 also covers radio and multiselect, and the export lists each candidate's vote fractions and a retired or needs_review status from the thresholds in the consensus_aggregation block. Neither the agreement metrics nor the consensus export checks display_logic, and an answer kept on a hidden question counts in the metrics and in the export. Set num_annotators_per_item to the number of annotators who vet each candidate.

Example

python potato/flask_server.py start examples/image/astronomy-real-bogus/config.yaml -p 8000

The example has four synthetic candidates, namely a supernova in a disk galaxy, a subtraction dipole, a cosmic ray, and a hostless point source. Each candidate has three 96 × 96 FITS images on one TAN WCS at 1″ per pixel and a light curve. The fourth candidate's images are gzipped. generate_data.py in the same folder rewrites the files.

Setting up a study with a coding agent

potato-skill 1.2.0 targets Potato 2.13.1 and knows the fits_cutout display by name. The skill guides the study-design side, including what to ask annotators, how many annotators vet each candidate, training and attention checks, and export. The skill does not yet document the FITS item format, the display options, or the astro extra. Install potato-skill, describe the study to your agent, and point the agent at the Markdown copy of this page for the formats and options. The Markdown copy is at https://www.potatoannotator.com/docs/annotation-types/astronomy.md.

Further reading

• Radio & Multiselect - The schemes that ask the vetting and tree questions
• Time Series - Marking transits and flares on a light curve
• Export Formats - Potato's export formats and the export command
• Inter-Annotator Agreement Explained - Choosing and reading an agreement coefficient
• How Many Annotators Do You Need? - Choosing annotators per item

For implementation details, see the source documentation.