Optimizing Local-Global Dependencies for Accurate 3D Human Pose Estimation

TL;DR

This paper introduces SSR-STF, a dual-stream transformer model that combines local and global features for improved 3D human pose estimation, achieving state-of-the-art results on benchmark datasets.

Contribution

The paper proposes SSRFormer with SSRA mechanism to effectively integrate local and global dependencies, enhancing pose estimation accuracy.

Findings

Achieves P1 errors of 37.4 mm on Human3.6M

Outperforms existing methods in accuracy and generalization

Effective in downstream tasks like human mesh recovery

Abstract

Transformer-based methods have recently achieved significant success in 3D human pose estimation, owing to their strong ability to model long-range dependencies. However, relying solely on the global attention mechanism is insufficient for capturing the fine-grained local details, which are crucial for accurate pose estimation. To address this, we propose SSR-STF, a dual-stream model that effectively integrates local features with global dependencies to enhance 3D human pose estimation. Specifically, we introduce SSRFormer, a simple yet effective module that employs the skeleton selective refine attention (SSRA) mechanism to capture fine-grained local dependencies in human pose sequences, complementing the global dependencies modeled by the Transformer. By adaptively fusing these two feature streams, SSR-STF can better learn the underlying structure of human poses, overcoming the limitations of traditional methods in local feature extraction. Extensive experiments on the Human3.6M and MPI-INF-3DHP datasets demonstrate that SSR-STF achieves state-of-the-art performance, with P1 errors of 37.4 mm and 13.2 mm respectively, outperforming existing methods in both accuracy and generalization. Furthermore, the motion representations learned by our model prove effective in downstream tasks such as human mesh recovery. Codes are available at https://github.com/poker-xu/SSR-STF.