3D Pose-Based Temporal Action Segmentation for Figure Skating: A Fine-Grained and Jump Procedure-Aware Annotation Approach

TL;DR

This paper introduces a new 3D pose dataset and annotation method for fine-grained temporal action segmentation in figure skating, demonstrating the effectiveness of 3D pose features for automatic jump procedure recognition.

Contribution

The study presents the FS-Jump3D dataset and a novel annotation approach for TAS in figure skating, addressing the lack of datasets and methods for 3D pose-based TAS tasks.

Findings

3D pose features improve TAS accuracy in figure skating

The FS-Jump3D dataset enables detailed jump procedure analysis

Fine-grained annotations enhance model learning for complex actions

Abstract

Understanding human actions from videos is essential in many domains, including sports. In figure skating, technical judgments are performed by watching skaters' 3D movements, and its part of the judging procedure can be regarded as a Temporal Action Segmentation (TAS) task. TAS tasks in figure skating that automatically assign temporal semantics to video are actively researched. However, there is a lack of datasets and effective methods for TAS tasks requiring 3D pose data. In this study, we first created the FS-Jump3D dataset of complex and dynamic figure skating jumps using optical markerless motion capture. We also propose a new fine-grained figure skating jump TAS dataset annotation method with which TAS models can learn jump procedures. In the experimental results, we validated the usefulness of 3D pose features as input and the fine-grained dataset for the TAS model in figure skating. FS-Jump3D Dataset is available at https://github.com/ryota-skating/FS-Jump3D.