Spatio-temporal MLP-graph network for 3D human pose estimation

TL;DR

This paper introduces a novel spatio-temporal MLP-graph network that leverages temporal data and adaptive graph modulation to improve 3D human pose estimation, especially under occlusions and ambiguities.

Contribution

It proposes a new architecture combining joint-mixing MLP blocks and graph weighted Jacobi networks with learnable modulation, advancing 3D pose estimation accuracy.

Findings

Outperforms recent state-of-the-art methods on benchmark datasets.

Effectively captures temporal correlations and joint relationships.

Demonstrates robustness to occlusions and ambiguous poses.

Abstract

Graph convolutional networks and their variants have shown significant promise in 3D human pose estimation. Despite their success, most of these methods only consider spatial correlations between body joints and do not take into account temporal correlations, thereby limiting their ability to capture relationships in the presence of occlusions and inherent ambiguity. To address this potential weakness, we propose a spatio-temporal network architecture composed of a joint-mixing multi-layer perceptron block that facilitates communication among different joints and a graph weighted Jacobi network block that enables communication among various feature channels. The major novelty of our approach lies in a new weighted Jacobi feature propagation rule obtained through graph filtering with implicit fairing. We leverage temporal information from the 2D pose sequences, and integrate weight modulation into the model to enable untangling of the feature transformations of distinct nodes. We also employ adjacency modulation with the aim of learning meaningful correlations beyond defined linkages between body joints by altering the graph topology through a learnable modulation matrix. Extensive experiments on two benchmark datasets demonstrate the effectiveness of our model, outperforming recent state-of-the-art methods for 3D human pose estimation.