Encoder-decoder with Multi-level Attention for 3D Human Shape and Pose Estimation

TL;DR

This paper introduces a Multi-level Attention Encoder-Decoder Network (MAED) that effectively models spatial, temporal, and joint-level relations to improve 3D human shape and pose estimation, especially in challenging scenarios.

Contribution

The novel MAED framework integrates multi-level attention mechanisms within an encoder-decoder architecture for enhanced 3D human pose estimation.

Findings

Outperforms previous methods on 3DPW, MPI-INF-3DHP, and Human3.6M benchmarks.

Achieves 6.2, 7.2, and 2.4 mm improvements in PA-MPJPE.

Effectively models multi-level relations for accurate pose estimation.

Abstract

3D human shape and pose estimation is the essential task for human motion analysis, which is widely used in many 3D applications. However, existing methods cannot simultaneously capture the relations at multiple levels, including spatial-temporal level and human joint level. Therefore they fail to make accurate predictions in some hard scenarios when there is cluttered background, occlusion, or extreme pose. To this end, we propose Multi-level Attention Encoder-Decoder Network (MAED), including a Spatial-Temporal Encoder (STE) and a Kinematic Topology Decoder (KTD) to model multi-level attentions in a unified framework. STE consists of a series of cascaded blocks based on Multi-Head Self-Attention, and each block uses two parallel branches to learn spatial and temporal attention respectively. Meanwhile, KTD aims at modeling the joint level attention. It regards pose estimation as a top-down hierarchical process similar to SMPL kinematic tree. With the training set of 3DPW, MAED outperforms previous state-of-the-art methods by 6.2, 7.2, and 2.4 mm of PA-MPJPE on the three widely used benchmarks 3DPW, MPI-INF-3DHP, and Human3.6M respectively. Our code is available at https://github.com/ziniuwan/maed.