Double-chain Constraints for 3D Human Pose Estimation in Images and Videos

TL;DR

This paper introduces a novel Double-chain Graph Convolutional Transformer model that effectively captures multi-level dependencies in 3D human pose estimation from images and videos, achieving state-of-the-art results.

Contribution

The paper proposes a double-chain design combining GCN and Transformer, along with modules for local and global constraints, to improve 3D human pose estimation accuracy.

Findings

Achieves state-of-the-art performance on Human3.6M and MPI-INF-3DHP datasets.

Effective integration of temporal information with minimal computational overhead.

Outperforms previous methods across all action categories in key datasets.

Abstract

Reconstructing 3D poses from 2D poses lacking depth information is particularly challenging due to the complexity and diversity of human motion. The key is to effectively model the spatial constraints between joints to leverage their inherent dependencies. Thus, we propose a novel model, called Double-chain Graph Convolutional Transformer (DC-GCT), to constrain the pose through a double-chain design consisting of local-to-global and global-to-local chains to obtain a complex representation more suitable for the current human pose. Specifically, we combine the advantages of GCN and Transformer and design a Local Constraint Module (LCM) based on GCN and a Global Constraint Module (GCM) based on self-attention mechanism as well as a Feature Interaction Module (FIM). The proposed method fully captures the multi-level dependencies between human body joints to optimize the modeling capability of the model. Moreover, we propose a method to use temporal information into the single-frame model by guiding the video sequence embedding through the joint embedding of the target frame, with negligible increase in computational cost. Experimental results demonstrate that DC-GCT achieves state-of-the-art performance on two challenging datasets (Human3.6M and MPI-INF-3DHP). Notably, our model achieves state-of-the-art performance on all action categories in the Human3.6M dataset using detected 2D poses from CPN, and our code is available at: https://github.com/KHB1698/DC-GCT.