KASportsFormer: Kinematic Anatomy Enhanced Transformer for 3D Human Pose Estimation on Short Sports Scene Video

TL;DR

KASportsFormer is a transformer-based framework that leverages kinematic anatomy features to improve 3D human pose estimation in short sports videos, effectively handling motion blur, occlusions, and instantaneous actions.

Contribution

The paper introduces a novel kinematic anatomy-informed transformer model specifically designed for sports scenarios, enhancing pose estimation accuracy in challenging short video clips.

Findings

Achieves state-of-the-art MPJPE errors of 58.0mm and 34.3mm on SportsPose and WorldPose datasets.

Effectively captures instantaneous sports motions with improved understanding of kinematic features.

Demonstrates robustness against motion blur, occlusions, and domain shifts in sports videos.

Abstract

Recent transformer based approaches have demonstrated impressive performance in solving real-world 3D human pose estimation problems. Albeit these approaches achieve fruitful results on benchmark datasets, they tend to fall short of sports scenarios where human movements are more complicated than daily life actions, as being hindered by motion blur, occlusions, and domain shifts. Moreover, due to the fact that critical motions in a sports game often finish in moments of time (e.g., shooting), the ability to focus on momentary actions is becoming a crucial factor in sports analysis, where current methods appear to struggle with instantaneous scenarios. To overcome these limitations, we introduce KASportsFormer, a novel transformer based 3D pose estimation framework for sports that incorporates a kinematic anatomy-informed feature representation and integration module. In which the inherent kinematic motion information is extracted with the Bone Extractor (BoneExt) and Limb Fuser (LimbFus) modules and encoded in a multimodal manner. This improved the capability of comprehending sports poses in short videos. We evaluate our method through two representative sports scene datasets: SportsPose and WorldPose. Experimental results show that our proposed method achieves state-of-the-art results with MPJPE errors of 58.0mm and 34.3mm, respectively. Our code and models are available at: https://github.com/jw0r1n/KASportsFormer