Skip to content
Showcase/Fis-V: Figure Skating Video Scoring
advancedvideo

Fis-V: Figure Skating Video Scoring

Automatic figure skating scoring as video regression. Annotators mark the technical elements and record the Total Element Score (TES) and Total Program Component Score (PCS) for a ladies' short program (Xu et al., IEEE TCSVT 2020).

About this dataset

Fis-V is a dataset for automatically scoring figure skating performances, introduced by Chengming Xu, Yanwei Fu, and colleagues (IEEE Transactions on Circuits and Systems for Video Technology, 2020). It frames scoring as video regression: predict the referee scores a skater would receive from the full performance video.

The videos are ladies' singles short-program performances from international competitions. Each video is annotated with two official referee scores: the Total Element Score (TES), reflecting the difficulty and execution of technical elements, and the Total Program Component Score (PCS), reflecting skating skills, transitions, performance, composition, and music interpretation. Scores are the official values from a panel of nine referees.

The dataset contains 500 videos from 149 professional skaters, each roughly two minutes fifty seconds long. Every video carries the two score labels (TES and PCS); the paper also evaluates on the earlier MIT-Skate dataset.

The Potato config below reproduces the scoring task with a video segment scheme for the technical elements plus fields for the Total Element Score and Total Program Component Score. Use it to collect human scores for figure skating or as a template for scored-sport video assessment.

Released
IEEE TCSVT 2020
Sport
Figure skating (ladies' short program)
Videos
500
Skaters
149
Score labels
TES + PCS (per video)
Referees
9
Frame 847 / 3200Running01:12 - 01:28Segments:WalkRunStandActionWalkRunStandWalkSceneOutdoorIndoorDrag to create and label temporal segments

Configuration Fileconfig.yaml

This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.

yaml
# Fis-V - Figure Skating Video Scoring
# Based on Xu, Fu, Zhang, Chen, Jiang & Xue, IEEE TCSVT 2020
# Paper: https://arxiv.org/abs/1802.02774
# Dataset: https://github.com/loadder/MS_LSTM
#
# Fis-V frames figure skating scoring as video regression: predict the
# referee scores a skater would receive from the full performance video.
# Each ladies' singles short-program video carries two official scores:
# the Total Element Score (TES) and the Total Program Component Score (PCS).
#
# This Potato config reproduces the scoring: watch the performance, mark the
# technical elements, and record the TES and PCS.

annotation_task_name: "Fis-V - Figure Skating Video Scoring"
task_dir: "."

data_files:
  - sample-data.json

item_properties:
  id_key: "id"
  text_key: "video_url"

output_annotation_dir: "annotation_output/"
output_annotation_format: "json"

annotation_schemes:
  # Step 1: mark the technical elements on the timeline
  - name: "elements"
    description: "Mark each technical element (jump, spin, step sequence) on the timeline."
    annotation_type: "video_annotation"
    mode: "segment"
    labels:
      - name: "jump"
        color: "#6366F1"
        key_value: "j"
      - name: "spin"
        color: "#14B8A6"
        key_value: "s"
      - name: "step_sequence"
        color: "#F59E0B"
        key_value: "t"
    frame_stepping: true
    show_timecode: true
    playback_rate_control: true
    video_fps: 25

  # Step 2: Total Element Score (TES)
  - annotation_type: text
    name: total_element_score
    description: "Total Element Score (TES) — difficulty and execution of technical elements."

  # Step 3: Total Program Component Score (PCS)
  - annotation_type: text
    name: total_program_component_score
    description: "Total Program Component Score (PCS) — skating skills, transitions, performance, composition, interpretation."

  # Step 4: notes on the elements performed
  - annotation_type: text
    name: element_notes
    description: "Optional notes on the elements performed and their quality."

annotation_instructions: |
  You will see a ladies' singles short-program figure skating performance.

  1. Mark each technical element on the timeline (jump, spin, step sequence).
  2. Record the Total Element Score (TES), which reflects the difficulty and
     execution of the technical elements.
  3. Record the Total Program Component Score (PCS), which reflects skating
     skills, transitions, performance, composition, and music interpretation.
  4. Optionally add notes on the elements performed.

  In the official competition these two scores come from a panel of nine
  referees; here you record the values that summarize the performance.

html_layout: |
  <div style="padding: 15px; max-width: 800px; margin: auto;">
    <div style="background: #eef2ff; border: 1px solid #c7d2fe; border-radius: 8px; padding: 10px 14px; margin-bottom: 12px; font-size: 13px; color: #3730a3;">
      Skating performance {{clip_id}} — mark the elements and record TES and PCS.
    </div>
    <video controls style="width: 100%; border-radius: 8px;" src="{{video_url}}"></video>
  </div>

allow_all_users: true
instances_per_annotator: 40
annotation_per_instance: 2
allow_skip: true
skip_reason_required: false

Sample Datasample-data.json

json
[
  {
    "id": "fisv_001",
    "clip_id": "WC-SP-014",
    "video_url": "https://example.com/fisv/clip_001.mp4",
    "text": "Ladies' short program: opening triple-triple combination jump."
  },
  {
    "id": "fisv_002",
    "clip_id": "GPF-SP-032",
    "video_url": "https://example.com/fisv/clip_002.mp4",
    "text": "Ladies' short program: layback spin into a step sequence."
  }
]

// ... and 3 more items

Get This Design

View on GitHub

Clone or download from the repository

Quick start:

git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/video/action-recognition/fisv-figure-skating-scoring
potato start config.yaml

Dataset & paper

Xu et al., IEEE TCSVT 2020

Citation (BibTeX)

bibtex
@article{xu2020learning,
    title={Learning to Score Figure Skating Sport Videos},
    author={Xu, Chengming and Fu, Yanwei and Zhang, Bing and Chen, Zitian and Jiang, Yu-Gang and Xue, Xiangyang},
    journal={IEEE Transactions on Circuits and Systems for Video Technology},
    volume={30},
    number={12},
    pages={4578--4590},
    year={2020},
    doi={10.1109/TCSVT.2019.2927118}
}

Details

Annotation Types

video_annotationtext

Domain

Computer VisionVideo UnderstandingSports

Use Cases

Action Quality AssessmentVideo RegressionSports Scoring

Tags

fis-vfigure-skatingaction-quality-assessmentvideo-regressionsportsscoring

Found an issue or want to improve this design?

Open an Issue