Skip to content
advancedimage

Open Images Image Classification

Large-scale image-level labeling following the Open Images dataset (Kuznetsova et al., IJCV 2020). Assign multi-label image-level tags across common object categories, and note occlusion, truncation, grouping, and visual relationships.

Labels:outdoornatureurbanpeopleanimal+−

Configuration Fileconfig.yaml

This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.

yaml
# Open Images Image Classification Configuration
# Based on Kuznetsova et al., IJCV 2020

annotation_task_name: "Open Images Image Classification"
task_dir: "."

data_files:
  - "sample-data.json"

item_properties:
  id_key: "id"
  text_key: "image_url"
  context_key: "context"

user_config:
  allow_all_users: true

annotation_schemes:
  - annotation_type: "multiselect"
    name: "object_classes"
    description: "Select all object classes visible in the image"
    labels:
      - name: "person"
        tooltip: "Human figure"
      - name: "car"
        tooltip: "Automobile"
      - name: "chair"
        tooltip: "Chair or seat"
      - name: "table"
        tooltip: "Table or desk"
      - name: "dog"
        tooltip: "Dog"
      - name: "cat"
        tooltip: "Cat"
      - name: "tree"
        tooltip: "Tree"
      - name: "building"
        tooltip: "Building structure"
      - name: "food"
        tooltip: "Food items"
      - name: "clothing"
        tooltip: "Clothing items"
      - name: "book"
        tooltip: "Book"
      - name: "phone"
        tooltip: "Phone"

  - annotation_type: "radio"
    name: "is_occluded"
    description: "Is the primary object occluded?"
    labels:
      - name: "not_occluded"
        tooltip: "Object is fully visible"
      - name: "partially_occluded"
        tooltip: "Object is partially hidden"
      - name: "heavily_occluded"
        tooltip: "Object is mostly hidden"

  - annotation_type: "radio"
    name: "is_truncated"
    description: "Is the object truncated by image boundary?"
    labels:
      - name: "not_truncated"
        tooltip: "Object is fully within frame"
      - name: "truncated"
        tooltip: "Object extends beyond image"

  - annotation_type: "radio"
    name: "is_group"
    description: "Is this a group of objects?"
    labels:
      - name: "single"
        tooltip: "Single object instance"
      - name: "group"
        tooltip: "Group of similar objects"

  - annotation_type: "text"
    name: "visual_relationships"
    description: "Describe visual relationships (e.g., 'person holding cup', 'dog on sofa')"

interface_config:
  item_display_format: "<img src='{{text}}' style='max-width:100%; max-height:500px;'/><br/><small>{{context}}</small>"

output_annotation_format: "json"
output_annotation_dir: "annotations"

Sample Datasample-data.json

json
[
  {
    "id": "oi_001",
    "image_url": "https://upload.wikimedia.org/wikipedia/commons/thumb/4/4f/Felis_silvestris_catus_lying_on_rice_straw.jpg/1200px-Felis_silvestris_catus_lying_on_rice_straw.jpg",
    "context": "Annotate all visible objects. Note occlusion, truncation, and any visual relationships between objects."
  },
  {
    "id": "oi_002",
    "image_url": "https://upload.wikimedia.org/wikipedia/commons/thumb/1/18/Dog_Breeds.jpg/1200px-Dog_Breeds.jpg",
    "context": "Identify objects and mark whether they are groups or single instances."
  }
]

// ... and 1 more items

Get This Design

View on GitHub

Clone or download from the repository

Quick start:

git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/image/classification/open-images
potato start config.yaml

Dataset & paper

Kuznetsova et al., IJCV 2020

Official dataset ↗Read the paper ↗

Citation (BibTeX)

bibtex
@article{kuznetsova2020open,
    title = "The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale",
    author = "Kuznetsova, Alina  and Rom, Hassan  and Alldrin, Neil  and Uijlings, Jasper  and Krasin, Ivan  and Pont-Tuset, Jordi  and Kamali, Shahab  and Popov, Stefan  and Malloci, Matteo  and Kolesnikov, Alexander  and Duerig, Tom  and Ferrari, Vittorio",
    journal = "International Journal of Computer Vision",
    volume = "128",
    number = "7",
    pages = "1956--1981",
    year = "2020",
    publisher = "Springer",
    url = "https://link.springer.com/article/10.1007/s11263-020-01316-z"
}

Details

Annotation Types

multiselectradiotext

Domain

Computer VisionImage Classification

Use Cases

Image ClassificationMulti-Label ClassificationVisual Relationships

Tags

open-imagesimage-classificationmulti-labelrelationshipslarge-scaleijcv2020

Found an issue or want to improve this design?

Open an Issue

Related Designs

CUB-200-2011: Fine-Grained Bird Species Dataset

CUB-200-2011 is a fine-grained bird image dataset of 11,788 photos across 200 species, each labeled with parts, attributes, and a bounding box (Wah et al., 2011). This Potato config reproduces the labeling task.

multiselectradio

FLAIR: French Land Cover from Aerospace Imagery

Land use and land cover classification from high-resolution aerial imagery. Annotators classify the primary land use category of aerial image patches and identify any secondary land uses present. Based on the FLAIR dataset from the French National Institute of Geographic and Forest Information (IGN).

multiselectradio

iWildCam Wildlife Detection & Classification

Camera trap image classification for wildlife monitoring (Beery et al., CVPR 2019). Classify wildlife species from camera trap images across diverse ecosystems worldwide.

multiselectradio