The Way Up: A Dataset for Hold Usage Detection in Sport Climbing

TL;DR

This paper introduces a new annotated dataset of climbing videos for hold usage detection and evaluates pose-estimation models to advance AI-assisted climbing analysis.

Contribution

It provides the first detailed climbing dataset with hold usage annotations and assesses pose-estimation models for detecting hold usage in sport climbing.

Findings

Keypoint-based models can detect hold usage with moderate accuracy.

Climbing-specific challenges affect pose-estimation performance.

The dataset enables future research in AI-assisted climbing systems.

Abstract

Detecting an athlete's position on a route and identifying hold usage are crucial in various climbing-related applications. However, no climbing dataset with detailed hold usage annotations exists to our knowledge. To address this issue, we introduce a dataset of 22 annotated climbing videos, providing ground-truth labels for hold locations, usage order, and time of use. Furthermore, we explore the application of keypoint-based 2D pose-estimation models for detecting hold usage in sport climbing. We determine usage by analyzing the key points of certain joints and the corresponding overlap with climbing holds. We evaluate multiple state-of-the-art models and analyze their accuracy on our dataset, identifying and highlighting climbing-specific challenges. Our dataset and results highlight key challenges in climbing-specific pose estimation and establish a foundation for future research toward AI-assisted systems for sports climbing.