FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessment

TL;DR

This paper introduces FineDiving, a detailed dataset with annotations for diving actions, and a procedure-aware model that improves action quality assessment by understanding internal temporal structures, leading to more accurate and interpretable scores.

Contribution

The paper presents a new fine-grained dataset and a novel procedure-aware approach with Temporal Segmentation Attention for more reliable and interpretable action quality assessment.

Findings

Achieves significant improvements over state-of-the-art methods.

Provides better interpretability through internal temporal structure understanding.

Demonstrates effectiveness on diverse diving events.

Abstract

Most existing action quality assessment methods rely on the deep features of an entire video to predict the score, which is less reliable due to the non-transparent inference process and poor interpretability. We argue that understanding both high-level semantics and internal temporal structures of actions in competitive sports videos is the key to making predictions accurate and interpretable. Towards this goal, we construct a new fine-grained dataset, called FineDiving, developed on diverse diving events with detailed annotations on action procedures. We also propose a procedure-aware approach for action quality assessment, learned by a new Temporal Segmentation Attention module. Specifically, we propose to parse pairwise query and exemplar action instances into consecutive steps with diverse semantic and temporal correspondences. The procedure-aware cross-attention is proposed to learn embeddings between query and exemplar steps to discover their semantic, spatial, and temporal correspondences, and further serve for fine-grained contrastive regression to derive a reliable scoring mechanism. Extensive experiments demonstrate that our approach achieves substantial improvements over state-of-the-art methods with better interpretability. The dataset and code are available at \url{https://github.com/xujinglin/FineDiving}.