STGFormer: Spatio-Temporal GraphFormer for 3D Human Pose Estimation in Video

TL;DR

STGFormer introduces a novel spatio-temporal graph attention framework that effectively models human body structure and dynamics, significantly improving 3D human pose estimation accuracy in videos.

Contribution

The paper proposes the Spatio-Temporal criss-cross Graph attention mechanism and a dual-path MHR-GCN to better capture spatiotemporal dependencies and higher-order information in video-based 3D pose estimation.

Findings

Achieves state-of-the-art results on Human3.6M dataset.

Outperforms previous methods on MPIINF-3DHP dataset.

Effectively models long-range spatiotemporal dependencies.

Abstract

The current methods of video-based 3D human pose estimation have achieved significant progress.However, they still face pressing challenges, such as the underutilization of spatiotemporal bodystructure features in transformers and the inadequate granularity of spatiotemporal interaction modeling in graph convolutional networks, which leads to pervasive depth ambiguity in monocular 3D human pose estimation. To address these limitations, this paper presents the Spatio-Temporal GraphFormer framework (STGFormer) for 3D human pose estimation in videos. First, we introduce a Spatio-Temporal criss-cross Graph (STG) attention mechanism, designed to more effectively leverage the inherent graph priors of the human body within continuous sequence distributions while capturing spatiotemporal long-range dependencies. Next, we present a dual-path Modulated Hop-wise Regular GCN (MHR-GCN) to independently process temporal and spatial dimensions in parallel, preserving features rich in temporal dynamics and the original or high-dimensional representations of spatial structures. Furthermore, the module leverages modulation to optimize parameter efficiency and incorporates spatiotemporal hop-wise skip connections to capture higher-order information. Finally, we demonstrate that our method achieves state-of-the-art performance on the Human3.6M and MPIINF-3DHP datasets.