Skip to content
教程閱讀約需 18 分鐘

Review MegaDetector Results and Publish Them to GBIF

Run MegaDetector on camera-trap images, have two volunteers and an expert name the species in Potato, and export a Camtrap DP package that validates.

Potato Team
此頁面尚未提供您所選語言的版本，當前顯示英文版本。

Camera-trap projects that run MegaDetector (Beery et al., 2019) can follow this tutorial to have two volunteers name the species in every sequence in Potato 2.12.1, send each disagreement to an expert, and export a Camtrap DP package for GBIF. MegaDetector is an open-source model that draws a box around each animal, person, or vehicle in a camera-trap image, and MegaDetector's README states that the model "only finds animals, it doesn't identify them to species level." Camtrap DP is the TDWG data exchange format for camera-trap data, with tables of deployments, media, and observations, and GBIF processes Camtrap DP packages into occurrence records. The worked example runs MegaDetector on 99 public images from two cameras in Colombia and ends with a package that passes the Camtrap DP validator. Camera Traps documents every import and export option.

The walkthrough ran on Potato 2.12.1, and the install command pins 2.12.1, because the preparation script in the GBIF section reads 2.12.1's exports. Potato 2.13.1 changes the exports and the image review, and the sections below say how where each change applies.

The images and the answer key

The images come from Orinoquía Camera Traps on LILA BC, 104,782 images from cameras in the El Rey Zamuro and Las Unamas reserves in Meta, Colombia, released under the Community Data License Agreement, permissive variant (CDLA-Permissive-1.0). LILA asks users of the dataset to cite Vélez et al. (2023), an evaluation of AI platforms for camera-trap data. The evaluation found high precision and recall for MegaDetector in separating blank images from images of animals, and generally low to moderate recall for the species classifications of three other platforms. The test run took 33 three-image sequences from cameras M04 and N29, 99 images in all, with 23 sequences of animals and 10 empty sequences. Every image in the dataset has an expert species label, and those labels served as the answer key for the reviewers in the test run.

LILA serves each image at https://storage.googleapis.com/public-datasets-lila/orinoquia-camera-traps/public/ followed by the image's path, such as N29/105EK113/04110961.JPG. Put the images in a folder named traps/ and keep the paths. The first folder in each path then names the camera.

Running MegaDetector

The command below installs MegaDetector, downloads MegaDetector 5a on the first run (281 MB), and writes one results file for the folder:

bash
pip install megadetector
python -m megadetector.detection.run_detector_batch MDV5A traps/ md_output.json \
    --recursive --output_relative_filenames --include_image_timestamp --threshold 0.05

--output_relative_filenames writes paths relative to traps/, and --include_image_timestamp copies each image's EXIF time into a datetime field. --threshold 0.05 keeps detections down to MegaDetector's conservative threshold. The import below applies the review's own cut-off, and trying another cut-off later needs no second MegaDetector run. The megadetector package 10.0.28 processed the 99 images in 36 seconds on an Apple silicon Mac and wrote batch-output format 1.6 with md_v5a.0.1.pt as the detector. The file records MegaDetector's typical threshold of 0.2 under detector_metadata. At that threshold, MegaDetector flagged 63 of the 64 images with an animal, put a box on 1 of the 35 empty images, and kept 80 boxes.

The dataset's metadata gives each time with a +00:00 offset, but the pacas in the dataset are active from 18:00 to 05:00 and people from 06:00 to 18:00. The cameras therefore recorded local time, five hours behind UTC, and the export step below adds that offset.

Adding cameras and sequences

MegaDetector's output names each image and lists its boxes, with no camera or sequence. Potato's import reads location, seq_id, and frame_num when an image entry has those fields. The script below takes the camera from the first folder of each path and starts a new sequence after a gap of more than 60 seconds:

python
import json
from collections import defaultdict
from datetime import datetime
 
results = json.load(open("md_output.json"))
 
# The first folder in each path is the camera, as in N29/105EK113/04110961.JPG
by_camera = defaultdict(list)
for image in results["images"]:
    image["location"] = image["file"].split("/")[0]
    by_camera[image["location"]].append(image)
 
# A new sequence starts when more than 60 seconds pass between two images
for camera, images in by_camera.items():
    images.sort(key=lambda im: im["datetime"])
    previous, n = None, 0
    for image in images:
        taken = datetime.strptime(image["datetime"], "%Y:%m:%d %H:%M:%S")
        if previous is None or (taken - previous).total_seconds() > 60:
            n += 1
            frame = 0
        image["seq_id"] = f"{camera}-{n:04d}"
        image["frame_num"] = frame
        frame += 1
        previous = taken
 
json.dump(results, open("md_output_seq.json", "w"), indent=1)
print(len(results["images"]), "images in",
      len({im["seq_id"] for im in results["images"]}), "sequences")

The script printed 99 images in 33 sequences, and the 33 sequences matched the dataset's own sequence IDs.

Importing the results into Potato

A camera-trap sequence is the unit ecologists count, because a peccary that walks past the camera in three frames is one detection. Install Potato and import the file with one item per sequence:

bash
pip install potato-annotation==2.12.1
potato import -i md_output_seq.json --image-dir traps --md-threshold 0.2 --sequences -o camera-trap-review/
text
Imported 33 item(s), 1 question(s), 0 answer(s) from 0 annotator(s), and 33 model suggestion(s) from MegaDetector batch output.
  left out: 19 detections below --md-threshold 0.2
  wrote camera-trap-review/data/md_output_seq.jsonl
  wrote camera-trap-review/config.yaml
  wrote camera-trap-review/README.md

Each sequence became one item, shown as a gallery of the three frames, and the generated config serves the images from traps/ without copying the images. The import writes one question, triage. Potato pre-fills the answer from the most confident detection in the sequence, or with empty when no frame has a detection at 0.2 or more. The pre-filled answer matched the answer key on all 33 sequences. MegaDetector names no species. The import therefore writes no species question, and the config below adds one.

The review config

The config below makes four changes to the config the import wrote. A species question lists the species on the site list by scientific name, because the scientific name is what Camtrap DP and Darwin Core record. humanize_labels: false keeps the names as written, since Potato otherwise title-cases a label and shows "Pecari Tajacu". Triage gets an unidentified option, for the reason in the next section. num_annotators_per_item sends every sequence to two reviewers and raises a sequence to three reviewers when the first two disagree. The config is complete and passes python -m potato.validate_cli config.yaml --strict on 2.12.1:

yaml
annotation_task_name: Orinoquia camera traps, sequence review
task_dir: .
output_annotation_dir: annotation_output/
data_files:
  - data/md_output_seq.jsonl
item_properties:
  id_key: id
  text_key: seq_id
user_config:
  allow_all_users: true
  users: []
media_directory: /data/orinoquia/traps
 
instance_display:
  fields:
    - key: frames
      type: gallery
      label: Sequence
      display_options:
        layout: grid
        columns: 3
        show_captions: true
        zoomable: true
 
# Two volunteers review every sequence, and a sequence they disagree on goes to a third reviewer
num_annotators_per_item:
  default: 2
  adaptive:
    enabled: true
    disagreement_threshold: 0.5
    boost_to: 3
 
pre_annotation:
  enabled: true
  field: predictions
  allow_modification: true
 
annotation_schemes:
  - annotation_type: radio
    name: triage
    description: What does this sequence show?
    labels: [empty, animal, unidentified, person, vehicle]
    sequential_key_binding: true
 
  - annotation_type: multiselect
    name: species
    description: Which species appear in the sequence?
    humanize_labels: false
    labels:
      - Pecari tajacu
      - Tayassu pecari
      - Dasyprocta fuliginosa
      - Cuniculus paca
      - Nasua nasua
      - Tapirus terrestris
      - Mitu salvini
      - Myrmecophaga tridactyla
      - Leopardus pardalis
      - Bos taurus

The import writes the full path of --image-dir as media_directory, shown here as /data/orinoquia/traps. The ten species are the seven in the example plus giant anteater, ocelot, and cattle. The three added species appear elsewhere in the Orinoquía set. All ten names matched an accepted name in the GBIF backbone taxonomy.

Volunteer review and the expert check

Start the server from the project folder:

bash
cd camera-trap-review
potato start config.yaml -p 8000

Each volunteer registers with a username and password, checks the triage answer with the keys 1 to 5, and ticks the species. A volunteer who sees an animal and cannot name the species chooses unidentified and ticks nothing. The option matters in Potato 2.12.1, because 2.12.1 compares two answers to a question only when both reviewers gave one. A sequence on which one volunteer ticked Nasua nasua and the other ticked nothing has no species answer to compare, and both triage answers read animal. The image review further down was first run without the option. In that run, the two volunteers disagreed on 13 images, and Potato raised only 5 of the 13 to a third reviewer. Potato 2.13.1 counts a blank as a different answer once the reviewer has left the item, so the option is no longer needed for escalation, and the option still gives volunteers a way to say they could not name the animal.

The expert logs in once both volunteers have finished. Every other sequence already has its two reviewers, so the server offers the expert the disagreements and nothing else. With more than two volunteers, a volunteer who has not seen a raised sequence can take the third place before the expert does. Potato compares the answers on every automatic save, and the page saves half a second after each click. A volunteer who ticks the wrong species and corrects the tick still sends the sequence to the expert. Such a correction costs the expert one extra sequence.

In the test run, two scripted volunteers misnamed the species in 15% and 10% of animal sequences. A misnamed answer was a look-alike species or unidentified. The first volunteer erred on 6 sequences and the second on 4. The volunteers disagreed on 8 sequences, Potato raised those 8 sequences to three reviewers, and the expert received the same 8. On one sequence of coatis, both volunteers chose unidentified, and the sequence never reached the expert. Two volunteers who make the same mistake agree, so check a random sample of agreed sequences as well.

Exporting the package and the occurrences

Export a Camtrap DP package and a Darwin Core occurrence table into one folder:

bash
python -m potato.export -c config.yaml -f camtrap_dp -o package/ \
    --option timezone=-05:00 --option contributor="Your Name" \
    --option sampling_design=systematicRandom --option capture_method=activityDetection
python -m potato.export -c config.yaml -f darwin_core -o package/ --option timezone=-05:00

The first command wrote datapackage.json, deployments.csv, media.csv, and observations.csv, and the second wrote occurrence.csv. timezone=-05:00 adds Colombia's UTC offset to the camera times. The Orinoquía cameras sat 1 km apart, and the test run used systematicRandom for that layout. Choose the sampling design from the Camtrap DP list that matches your own survey. Each sequence gives event-level observations, with the sequence ID as eventID and the first and last frame times as eventStart and eventEnd. The export warned that cameras M04 and N29 have no coordinates. Camtrap DP requires coordinates for every deployment. Leave --option license off, because the option writes only a data license, and the script in the next section writes the two licenses the profile asks for.

The two tables are not ready to publish, because Potato 2.12.1 writes every reviewer's answer. observations.csv had 74 rows for 33 sequences, one per sequence from each volunteer and one more from the expert on 8 sequences. occurrence.csv had 46 occurrences for the 23 sequences of animals. The export left out 8 unidentified answers, since Darwin Core needs a scientific name. Published as exported, the package would count each animal two or three times. --option annotator=<user> does not help here, since the camtrap_dp and darwin_core exports ignore the option, and only the megadetector export reads the option.

Potato 2.13.1 writes one reviewer per sequence in both tables, the first by user ID unless --option annotator=<user> names a reviewer, and warns about the sequences where the reviewers disagree. Run on this page's saved review, 2.13.1 wrote 33 observations and 18 occurrences, all from the first volunteer, so the expert's answers on the 8 disputed sequences were not in the tables. A 2.13.1 export therefore still needs the expert's answers merged in, which the script below does for 2.12.1's tables. In 2.13.1, --option license= and --option media_license= write the two licenses the profile asks for, the package uses the Camtrap DP 1.0.2 profile, and --option media_base_url= turns Darwin Core's associatedMedia into public URLs. The coordinates still go in by hand.

Preparing the package for GBIF

The script below keeps one reviewer's answer per sequence. The script keeps the expert's answer where the expert reviewed the sequence, and otherwise the answer of the volunteer whose username sorts first, since the two volunteers agreed on every sequence the expert did not see. Potato writes each observationID and occurrenceID as the item, the reviewer, and a number joined by colons, and the script reads the reviewer from those IDs. The script then adds the facts Potato does not have. Each camera's coordinates go into deployments.csv, occurrence.csv, and the package's spatial box. A data license and a media license go into datapackage.json, since the Camtrap DP profile requires both scopes once any license is listed. Public image URLs replace the local paths. Run the script in camera-trap-review/:

python
import csv
import json
from collections import defaultdict
 
EXPERT = "expert"
# Each camera's latitude and longitude in decimal degrees (WGS84)
CAMERAS = {"M04": (4.05, -71.95), "N29": (4.05, -71.95)}
# Where the images are public, if they are; "" keeps the local paths
MEDIA_BASE = "https://storage.googleapis.com/public-datasets-lila/orinoquia-camera-traps/public/"
DATA_LICENSE = "CC-BY-4.0"
MEDIA_LICENSE = "CDLA-Permissive-1.0"
 
 
def read(path):
    with open(path, newline="", encoding="utf-8") as f:
        reader = csv.DictReader(f)
        return reader.fieldnames, list(reader)
 
 
def write(path, columns, rows):
    with open(path, "w", newline="", encoding="utf-8") as f:
        writer = csv.DictWriter(f, fieldnames=columns)
        writer.writeheader()
        writer.writerows(rows)
 
 
def one_reviewer(rows, id_column):
    # Potato writes each ID as <item>:<reviewer>:<n>
    reviewers = defaultdict(set)
    for row in rows:
        item, reviewer, _ = row[id_column].rsplit(":", 2)
        reviewers[item].add(reviewer)
    # The expert's answer where the expert saw the item, otherwise the first volunteer's
    keep = {item: EXPERT if EXPERT in names else min(names) for item, names in reviewers.items()}
    return [row for row in rows
            if row[id_column].rsplit(":", 2)[1] == keep[row[id_column].rsplit(":", 2)[0]]]
 
 
columns, rows = read("package/observations.csv")
kept = one_reviewer(rows, "observationID")
write("package/observations.csv", columns, kept)
print("observations:", len(rows), "->", len(kept))
 
columns, rows = read("package/occurrence.csv")
kept = one_reviewer(rows, "occurrenceID")
for row in kept:
    lat, lon = CAMERAS[row["locationID"]]
    row.update(decimalLatitude=lat, decimalLongitude=lon, geodeticDatum="EPSG:4326")
    if MEDIA_BASE:
        row["associatedMedia"] = row["associatedMedia"].replace("/media/", MEDIA_BASE)
write("package/occurrence.csv", columns, kept)
print("occurrences:", len(rows), "->", len(kept))
 
columns, rows = read("package/deployments.csv")
for row in rows:
    row["latitude"], row["longitude"] = CAMERAS[row["deploymentID"]]
write("package/deployments.csv", columns, rows)
 
if MEDIA_BASE:
    columns, rows = read("package/media.csv")
    for row in rows:
        row["filePath"], row["filePublic"] = MEDIA_BASE + row["filePath"], "true"
    write("package/media.csv", columns, rows)
 
with open("package/datapackage.json", encoding="utf-8") as f:
    package = json.load(f)
lats = [lat for lat, _ in CAMERAS.values()]
lons = [lon for _, lon in CAMERAS.values()]
w, s, e, n = min(lons), min(lats), max(lons), max(lats)
package["spatial"] = {"type": "Polygon", "bbox": [w, s, e, n],
                      "coordinates": [[[w, s], [e, s], [e, n], [w, n], [w, s]]]}
package["licenses"] = [{"name": DATA_LICENSE, "scope": "data"},
                       {"name": MEDIA_LICENSE, "scope": "media"}]
# 1.0.2 has the same terms as 1.0, and current frictionless can read its profile
text = json.dumps(package, indent=2).replace("/camtrap-dp/1.0/", "/camtrap-dp/1.0.2/")
with open("package/datapackage.json", "w", encoding="utf-8") as f:
    f.write(text)

The script cut the observations from 74 to 33 and the occurrences from 46 to 22. The 33 observations are 22 named animals, 10 blanks, and the coati sequence both volunteers left unidentified, and 32 of the 33 match the answer key. Replace EXPERT with your expert's username and CAMERAS with each camera's own coordinates. The Orinoquía dataset publishes no camera coordinates, and the test run used one placeholder point for both cameras. GBIF's IPT manual asks for CC0, CC BY, or CC BY-NC on published data and recommends version 4.0 of the last two, so choose DATA_LICENSE from those three licenses.

The last line of the script changes the profile version. Potato 2.12.1 writes the Camtrap DP 1.0 profile URL. Frictionless 5.17 and later cannot resolve one reference in the 1.0 profile, and frictionless 5.21 stopped with the error InvalidAnchor: '#$defs/version' is not a valid anchor. Camtrap DP 1.0.1 fixed the reference without changing any term, and 1.0.2 is the current release. Validate the package with the command the Camtrap DP site gives:

bash
pip install frictionless
frictionless validate package/datapackage.json

Frictionless reported all three tables of the prepared package as VALID. The same command on the package as exported failed on the profile URL and then on the empty spatial box. GBIF reads a Camtrap DP package with the camtrapdp R package, and camtrapdp builds occurrences from event-level observations by default. The sequence review writes event-level observations. Version 0.6.1 of camtrapdp read the prepared package, check_camtrapdp() raised no error, and write_dwc() wrote 23 occurrences. Of the 23, 22 name a species, and 1 records only Animalia for the coati sequence both volunteers left unidentified. The camtrapdp output gives MachineObservation as the basisOfRecord of all 23, where Potato's own darwin_core export writes HumanObservation for an identification a person made. Upload the package to a GBIF IPT, version 3 or later, through your GBIF publisher. The test run stopped at validation and sent nothing to GBIF.

Keeping the corrected boxes

Review single images instead of sequences when the project wants the boxes as well, for example to train a classifier on its own site. Import without --sequences and each image becomes one item with a detections box question, pre-drawn with MegaDetector's boxes. The image config adds the species names to the box labels as well as to the species question.

The image review differs from the sequence review in three ways. First, Potato 2.12.1 has no control that changes the label of an existing box, and Potato 2.13.1 relabels a selected box when the reviewer clicks a species label. In 2.12.1, a label button sets the label for the next box drawn, and a drag that starts inside MegaDetector's box moves the box. To name a species, select MegaDetector's box with V and a click, press Delete, choose the species, and draw a new box with R. Second, each image gives media-level observations, and an animal in three frames becomes three observations. Third, camtrapdp reads media-level observations only when datapackage.json sets "gbifIngestion": {"observationLevel": "media"}. Add the setting before validating, because camtrapdp turned the image package into 80 occurrences with the setting and into none without the setting. Potato 2.13.1 writes the setting itself on a package of single images.

The image run used the same volunteers and expert. Potato raised 13 of 99 images to the expert, and the export wrote 247 observations. The script above cut the 247 to 115. The 115 are 79 boxes with a species, 1 species tick on the image MegaDetector missed, and 35 blanks. A coati in three frames of one sequence is three of those observations, where the sequence review records the coati once.

python -m potato.export -c config.yaml -f megadetector -o export/ writes megadetector_reviewed.json in batch-output format 1.4, for tools that read MegaDetector results, with one reviewer's answer per item. The default reviewer is the first by user ID, and --option annotator=<user> names another.

Other tools for camera-trap review

AddaxAI runs MegaDetector and a regional species model on Windows, Linux, or an Apple silicon Mac with no internet connection after installation, and exports Camtrap DP and Timelapse recognition files. AddaxAI is free, some species models have non-commercial licenses, and the program was called EcoAssist before a rename. Timelapse is a Windows program for tagging images that imports AddaxAI and MegaDetector recognition files and draws the boxes. MegaDetector's README lists Wildlife Insights, TrapTagger, Agouti, and Camelot among the platforms that work with MegaDetector, and Wildlife Insights identifies the animals in uploaded photos with machine learning. Potato fits when several reviewers share the identifications in a browser, when each reviewer's answers are kept for agreement, and when the images stay on your own server. Camera trap and bioacoustics tools compared sets these tools beside each other, with prices.

Limits of Potato for publishing to GBIF

  • One reviewer per item in the GBIF exports. In Potato 2.13.1, camtrap_dp and darwin_core take one reviewer's answers for every item, the first by user ID or the reviewer --option annotator names, so an expert's answers on the disputed items are merged by hand. Potato 2.12.1 wrote one observation per reviewer.
  • Coordinates by hand. Camtrap DP needs each camera's coordinates. Potato 2.13.1 reads latitude and longitude from MegaDetector entries that include them, and Potato 2.12.1 read neither.
  • No species model. Potato runs no detector or classifier. A classifier's species in MegaDetector's classifications field become pre-filled labels on import.
  • No individual counts in sequence review. The event observations leave count empty.

Where to go next

References

  1. Beery, S., Morris, D., & Yang, S. (2019). Efficient pipeline for camera trap image review. arXiv. https://arxiv.org/abs/1907.06772
  2. Vélez, J., McShea, W., Shamon, H., Castiblanco-Camacho, P. J., Tabak, M. A., Chalmers, C., Fergus, P., & Fieberg, J. (2023). An evaluation of platforms for processing camera-trap data using artificial intelligence. Methods in Ecology and Evolution, 14(2), 459-477. https://doi.org/10.1111/2041-210X.14044