SMPLer: Taming Transformers for Monocular 3D Human Shape and Pose Estimation

TL;DR

SMPLer introduces a novel Transformer framework for monocular 3D human shape and pose estimation that effectively utilizes high-resolution features with reduced complexity, achieving state-of-the-art accuracy.

Contribution

The paper proposes a decoupled attention mechanism and SMPL-based representation, enabling efficient high-resolution feature utilization in Transformers for 3D human reconstruction.

Findings

Achieves MPJPE of 45.2 mm on Human3.6M dataset.

Outperforms Mesh Graphormer by over 10% in accuracy.

Uses fewer than one-third of the parameters of previous methods.

Abstract

Existing Transformers for monocular 3D human shape and pose estimation typically have a quadratic computation and memory complexity with respect to the feature length, which hinders the exploitation of fine-grained information in high-resolution features that is beneficial for accurate reconstruction. In this work, we propose an SMPL-based Transformer framework (SMPLer) to address this issue. SMPLer incorporates two key ingredients: a decoupled attention operation and an SMPL-based target representation, which allow effective utilization of high-resolution features in the Transformer. In addition, based on these two designs, we also introduce several novel modules including a multi-scale attention and a joint-aware attention to further boost the reconstruction performance. Extensive experiments demonstrate the effectiveness of SMPLer against existing 3D human shape and pose estimation methods both quantitatively and qualitatively. Notably, the proposed algorithm achieves an MPJPE of 45.2 mm on the Human3.6M dataset, improving upon Mesh Graphormer by more than 10% with fewer than one-third of the parameters. Code and pretrained models are available at https://github.com/xuxy09/SMPLer.