Skip to content
beginnerimage

Visual Question Answering

Answer questions about images for VQA dataset creation.

Get This Design

View on GitHub(opens in a new tab)

Clone or download from the repository

Quick start:

bash
git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/evaluation/visual-qa
potato start config.yaml

Configuration File

yaml
annotation_task_name: "Visual Question Answering"
task_name: "Visual Question Answering"
task_description: "Answer the question about the image."
task_dir: "."
port: 8000

data_files:
  - "sample-data.json"

item_properties:
  id_key: "id"
  text_key: "image_url"
  image_key: "image_url"
  context_key: "question"

annotation_schemes:
  - annotation_type: text
    name: answer
    description: "Provide a concise answer to the question"
    required: true

  - annotation_type: radio
    name: confidence
    description: "How confident are you in your answer?"
    labels:
      - "Very confident"
      - "Somewhat confident"
      - "Not confident"
    required: true

output_annotation_dir: "output/"
output_annotation_format: "json"

Sample Data

json
[
  {
    "id": "1",
    "image_url": "https://images.unsplash.com/photo-1560807707-8cc77767d783?w=640",
    "question": "What color is the dog?"
  },
  {
    "id": "2",
    "image_url": "https://images.unsplash.com/photo-1449824913935-59a10b8d2000?w=640",
    "question": "What time of day does this appear to be?"
  }
]

Details

Annotation Types

radiotext

Domain

Computer VisionNLP

Use Cases

Visual QAMultimodal

Tags

vqavisual-qamultimodalimage

Found an issue or want to improve this design?

Open an Issue(opens in a new tab)

Related Designs

TextVQA: Reading Text in Images for VQA

TextVQA is a visual question answering dataset where reading scene text in the image is needed to answer. This Potato config reproduces the task: show an image and question, then collect a free-text answer.

textradio

CUB-200-2011: Fine-Grained Bird Species Dataset

CUB-200-2011 is a fine-grained bird image dataset of 11,788 photos across 200 species, each labeled with parts, attributes, and a bounding box (Wah et al., 2011). This Potato config reproduces the labeling task.

multiselectradio

EPIC-KITCHENS Egocentric Action Annotation

Annotate fine-grained actions in egocentric kitchen videos with verb-noun pairs. Identify cooking actions from a first-person perspective.

radiotext