YouTube Highlights Detection
Detect highlight-worthy moments in domain-specific videos. Annotators identify the most engaging segments for automatic highlight generation.
Configuration Fileconfig.yaml
This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.
# YouTube Highlights Detection Configuration
# Based on Sun et al., ECCV 2014
# Task: Identify highlight-worthy moments in domain-specific videos
annotation_task_name: "YouTube Highlights Detection"
task_dir: "."
data_files:
- data.json
item_properties:
id_key: "id"
text_key: "video_url"
output_annotation_dir: "annotation_output/"
output_annotation_format: "json"
annotation_schemes:
- name: "highlights"
description: |
Mark the HIGHLIGHT moments in this video.
These are the parts viewers would most want to see.
annotation_type: "video_annotation"
mode: "segment"
labels:
- name: "highlight"
color: "#F59E0B"
key_value: "h"
- name: "best_moment"
color: "#EF4444"
key_value: "b"
frame_stepping: true
show_timecode: true
playback_rate_control: true
video_fps: 30
- name: "highlight_type"
description: "What type of highlight is the BEST moment in this video?"
annotation_type: radio
labels:
- "Skill/Trick - impressive technique"
- "Fail/Funny - entertaining mistake"
- "Scenic/Beautiful - visually stunning"
- "Dramatic - emotional peak"
- "Informative - key learning moment"
- "Social - interesting interaction"
- name: "domain_relevance"
description: "How relevant is this video to its category?"
annotation_type: radio
labels:
- "Highly relevant - typical example"
- "Relevant - fits the category"
- "Somewhat relevant - partial fit"
- "Not very relevant - miscategorized"
allow_all_users: true
instances_per_annotator: 50
annotation_per_instance: 3
annotation_instructions: |
## YouTube Highlights Detection
Identify the most highlight-worthy moments in each video.
### What is a highlight?
- The moment you'd share with a friend
- The part that makes the video worth watching
- Peak action, emotion, or visual interest
### Domain-specific guidance:
**Surfing**: Big waves, successful rides, wipeouts
**Skating**: Tricks landing, impressive moves, falls
**Gymnastics**: Difficult moves, perfect landings
**Parkour**: Creative moves, big jumps, close calls
**Dog**: Cute moments, tricks, funny behavior
### Guidelines:
- Mark ALL highlight-worthy segments
- Use "best_moment" for the single best highlight
- Highlights are typically 2-10 seconds
- Consider what would make a good thumbnail/preview
### Tips:
- Put yourself in the viewer's shoes
- Think about what would get the most engagement
- Domain expertise helps identify skill-based highlights
Sample Datasample-data.json
[
{
"id": "yt_highlight_001",
"video_url": "https://example.com/videos/surfing_clip.mp4",
"domain": "surfing",
"duration": 120
},
{
"id": "yt_highlight_002",
"video_url": "https://example.com/videos/skating_clip.mp4",
"domain": "skating",
"duration": 90
}
]Get This Design
Clone or download from the repository
Quick start:
git clone https://github.com/davidjurgens/potato-showcase.git cd potato-showcase/video/summarization/youtube-highlights potato start config.yaml
Dataset & paper
Sun et al., ECCV 2014
Citation (BibTeX)
@inproceedings{sun2014ranking,
title={Ranking domain-specific highlights by analyzing edited videos},
author={Sun, Min and Farhadi, Ali and Seitz, Steve},
booktitle={European Conference on Computer Vision},
pages={787--802},
year={2014}
}Details
Annotation Types
Domain
Use Cases
Tags
Found an issue or want to improve this design?
Open an IssueRelated Designs
DiDeMo Moment Retrieval
Localizing natural language descriptions to specific video moments. Given a text query, annotators identify the corresponding temporal segment in the video.
FineSports: Fine-grained Basketball Action Recognition
Fine-grained NBA basketball action recognition. Annotators track the ball-handler across a clip and label a two-level hierarchy of 12 coarse ball-handler actions and 52 fine-grained sub-actions (Xu et al., CVPR 2024).
Scene Boundary Detection
Identify scene boundaries in documentary and narrative videos. Annotators mark transitions between semantically coherent scenes based on visual, audio, and narrative cues.