Skip to content
Showcase/WTW Wired Table Structure Annotation
advancedimage

WTW Wired Table Structure Annotation

Cell-grid structure annotation for photographed and scanned wired tables, based on the WTW (Wired Table in the Wild) dataset (Long et al., ICCV 2021). Annotators set each table's row/column grid dimensions and mark every cell's logical role, including headers, empty cells, and spanned continuations of merged cells, then record which of WTW's seven challenging conditions apply and the image source. Pixel-level cell quadrilaterals from the full WTW protocol are deliberately out of scope.

About this dataset

WTW (Wired Table in the Wild) is the first large dataset for parsing the structure of wired tables photographed outside clean documents: on bottles, boxes, and signs as well as in archival and printed documents. It contains 14,581 images (10,970 training / 3,611 testing), sourced roughly 50% from natural scenes, 30% from archival documents, and 20% from printed documents.

The dataset deliberately covers seven challenging cases that break document-centric table parsers: inclined tables, curved tables, occluded or blurred tables, extreme-aspect-ratio tables, overlaid tables, multi-color tables, and irregular tables. Annotations include cell coordinates (following the ICDAR2019 benchmark's convention of localizing cells by the inner table lines), row/column information, and table instance labels for multi-table images.

This showcase task reproduces the logical half of that protocol with Potato's table_grid scheme: the annotator sets the grid dimensions, then clicks cells to cycle through roles (data, column header, row header, empty, spanned). The 'spanned' role marks grid positions covered by a merged cell starting above or to the left, which encodes each spanning cell's row/column start-end extent in the ICDAR-2019 style.

One deliberate simplification, flagged in the config header: WTW's full annotation also draws each cell's physical quadrilateral along the inner table lines, and pixel-level coordinates are not captured by table_grid. Companion schemes record the applicable challenging cases from the WTW taxonomy and the image source category, mirroring how the dataset itself is organized.

Unlike the page-layout tasks in this showcase (OmniDocBench document parsing, DocBank), which draw region boxes around whole tables and figures, this task targets the structure inside a table: the cell grid that region boxes cannot express.

Released
2021 (ICCV; arXiv:2109.02199)
Images
14,581 (10,970 train / 3,611 test)
Sources
~50% natural scenes / 30% archival / 20% printed
Challenging cases
7 (inclined, curved, occluded/blurred, extreme aspect ratio, overlaid, multi-color, irregular)
Cell convention
ICDAR2019 inner-table-line localization
Labels:outdoornatureurbanpeopleanimal+

Configuration Fileconfig.yaml

This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.

yaml
# WTW Wired Table Structure Annotation
# Based on: Rujiao Long, Wen Wang, Nan Xue, Feiyu Gao, Zhibo Yang, Yongpan
#   Wang, and Gui-Song Xia. "Parsing Table Structures in the Wild." IEEE/CVF
#   International Conference on Computer Vision (ICCV) 2021. arXiv:2109.02199.
# Paper: https://openaccess.thecvf.com/content/ICCV2021/html/Long_Parsing_Table_Structures_in_the_Wild_ICCV_2021_paper.html
# Dataset: https://github.com/wangwen-whu/WTW-Dataset
#
# Task: Annotate the cell structure of photographed and scanned wired tables
# from the WTW dataset (natural scenes, archival documents, and printed
# documents, covering seven challenging cases: inclined, curved, occluded or
# blurred, extreme aspect ratio, overlaid, multi-color, and irregular tables).
# Using the table_grid scheme, the annotator sets the table's row/column grid
# dimensions and clicks cells to mark their logical role: data, column header,
# row header, empty, or spanned (a grid position covered by a merged cell that
# starts above or to its left, which encodes the ICDAR-2019-style row/column
# start-end extent of spanning cells at the logical level). Deliberate
# simplification: WTW's full annotation also localizes each cell's physical
# quadrilateral along the inner table lines (following the ICDAR2019
# benchmark); pixel-level cell coordinates are not captured by table_grid, so
# this showcase covers the logical-structure half of the protocol only.
#
# Annotation instructions adapted from Section 3 (the WTW dataset) of the
# paper; the paper does not publish the verbatim annotator guidelines.

annotation_task_name: "WTW Wired Table Structure Annotation"
task_dir: "."

data_files:
  - sample-data.json

item_properties:
  id_key: "id"
  text_key: "image"

output_annotation_dir: "annotation_output/"
output_annotation_format: "json"

port: 8000
server_name: localhost

annotation_instructions: |
  ## Wired Table Structure Annotation

  You will see a photograph or scan of a table with visible (wired) borders,
  taken in the wild: on bottles and boxes, in archival documents, or in
  printed pages. Your job is to reconstruct the table's logical cell grid.

  ### 1. Set the grid size
  Count the rows and columns of the table as defined by its inner table lines
  (following the ICDAR2019 convention, cell extents are defined by the inner
  lines, not the outer border). Set the grid dimensions accordingly. A cell
  that spans several rows or columns still occupies all of those grid
  positions.

  ### 2. Mark cell roles
  Click each cell to cycle through its role:

  - **data** -- an ordinary content cell (the default; you do not need to
    mark these explicitly).
  - **col_header** -- a cell that labels the column(s) below it.
  - **row_header** -- a cell that labels the row(s) to its right.
  - **empty** -- a cell position with no content at all.
  - **spanned** -- a grid position covered by a merged cell that starts above
    it or to its left. For a cell spanning columns 2-3, mark position 2 with
    the cell's real role and position 3 as spanned; likewise for row spans.
    This records each spanning cell's row/column start and end extent.

  ### 3. Challenging cases
  Select every challenging condition that applies to this table image:
  inclined, curved, occluded or blurred, extreme aspect ratio, overlaid,
  multi-color, or irregular. Select none if the table is a plain upright
  scan.

  ### 4. Image source
  Finally, record whether the image is a natural-scene photograph, an
  archival document, or a printed document.

  Annotate the table exactly as printed, even if it contains errors. If the
  image contains more than one table, annotate the largest fully visible one
  and skip the item if that is ambiguous.

annotation_schemes:
  - annotation_type: table_grid
    name: structure
    description: "Set the grid size, then click cells to mark headers, empty cells, and spanned continuations."
    image_key: image
    rows_key: rows
    cols_key: cols
    roles: [data, col_header, row_header, empty, spanned]

  - annotation_type: multiselect
    name: challenging_cases
    description: "Select all challenging conditions that apply to this table image (per the WTW taxonomy)."
    labels:
      - name: "inclined"
        tooltip: "The table is rotated or tilted relative to the image frame"
        key_value: "1"
      - name: "curved"
        tooltip: "The table surface is bent or warped, e.g., on a bottle or curved page"
        key_value: "2"
      - name: "occluded-or-blurred"
        tooltip: "Parts of the table are hidden by objects, glare, or blur"
        key_value: "3"
      - name: "extreme-aspect-ratio"
        tooltip: "The table is unusually wide or tall relative to its other dimension"
        key_value: "4"
      - name: "overlaid"
        tooltip: "Text, stamps, or graphics are printed over the table lines"
        key_value: "5"
      - name: "multi-color"
        tooltip: "Cells or lines use multiple colors that complicate line detection"
        key_value: "6"
      - name: "irregular"
        tooltip: "Non-rectangular layout with unusual cell arrangements"
        key_value: "7"

  - annotation_type: radio
    name: image_source
    description: "What kind of image is this table from?"
    labels:
      - name: "natural-scene"
        tooltip: "A photograph of a table in the wild, e.g., on packaging, signs, or labels"
        key_value: "q"
      - name: "archival-document"
        tooltip: "A scan or photograph of a historical/archival document"
        key_value: "w"
      - name: "printed-document"
        tooltip: "A scan or screenshot of a modern printed or digital document"
        key_value: "e"

allow_all_users: true
instances_per_annotator: 25
annotation_per_instance: 2
allow_skip: true

Sample Datasample-data.json

json
[
  {
    "id": "wtw_001",
    "image": "https://example.com/images/wtw/nutrition_label_bottle.jpg",
    "rows": 8,
    "cols": 3,
    "table_context": "Nutrition facts table printed on a curved juice bottle label, photographed at an angle in a grocery store."
  },
  {
    "id": "wtw_002",
    "image": "https://example.com/images/wtw/archival_ledger_1912.jpg",
    "rows": 12,
    "cols": 5,
    "table_context": "Hand-ruled accounting ledger page from a 1912 municipal archive scan, with faded ink and a stamp overlaying the lower rows."
  }
]

// ... and 8 more items

Get This Design

View on GitHub

Clone or download from the repository

Quick start:

git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/image/wtw-table-structure-annotation
potato start config.yaml

Dataset & paper

Long et al., ICCV 2021

Citation (BibTeX)

bibtex
@inproceedings{Long_2021_ICCV,
  author={Long, Rujiao and Wang, Wen and Xue, Nan and Gao, Feiyu and Yang, Zhibo and Wang, Yongpan and Xia, Gui-Song},
  title={Parsing Table Structures in the Wild},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  month={October},
  year={2021}
}

Details

Annotation Types

table_gridmultiselectradio

Domain

Document AnalysisComputer Vision

Use Cases

Table Structure RecognitionOCR PipelinesScene Text Understanding

Tags

table-structuretable-gridwired-tablesdocument-parsingscene-imagesWTW

Found an issue or want to improve this design?

Open an Issue