Fine-grained Action Analysis: A Multi-modality and Multi-task Dataset of Figure Skating

TL;DR

This paper introduces MMFS, a comprehensive dataset for fine-grained figure skating action analysis, combining multiple modalities and tasks to enhance recognition and quality assessment.

Contribution

The paper presents a novel multi-modality, multi-task dataset with independent spatial and temporal categories, including skeleton data for detailed action analysis.

Findings

First dataset with independent spatial and temporal categories.

Inclusion of skeleton modality for action quality assessment.

Baseline methods demonstrate the dataset's utility for action recognition and quality evaluation.

Abstract

The fine-grained action analysis of the existing action datasets is challenged by insufficient action categories, low fine granularities, limited modalities, and tasks. In this paper, we propose a Multi-modality and Multi-task dataset of Figure Skating (MMFS) which was collected from the World Figure Skating Championships. MMFS, which possesses action recognition and action quality assessment, captures RGB, skeleton, and is collected the score of actions from 11671 clips with 256 categories including spatial and temporal labels. The key contributions of our dataset fall into three aspects as follows. (1) Independently spatial and temporal categories are first proposed to further explore fine-grained action recognition and quality assessment. (2) MMFS first introduces the skeleton modality for complex fine-grained action quality assessment. (3) Our multi-modality and multi-task dataset encourage more action analysis models. To benchmark our dataset, we adopt RGB-based and skeleton-based baseline methods for action recognition and action quality assessment.