DiffHPE: Robust, Coherent 3D Human Pose Lifting with Diffusion

TL;DR

DiffHPE introduces a diffusion model-based approach to 3D human pose estimation, significantly improving robustness, accuracy, and coherence, especially under occlusions and mismatched training-inference conditions.

Contribution

The paper presents DiffHPE, the first integration of diffusion models into 3D human pose estimation, enhancing robustness and accuracy over existing supervised methods.

Findings

DiffHPE outperforms existing models on Human 3.6M dataset.

Diffusion models improve robustness against occlusions.

Combining diffusion with supervised models yields superior results.

Abstract

We present an innovative approach to 3D Human Pose Estimation (3D-HPE) by integrating cutting-edge diffusion models, which have revolutionized diverse fields, but are relatively unexplored in 3D-HPE. We show that diffusion models enhance the accuracy, robustness, and coherence of human pose estimations. We introduce DiffHPE, a novel strategy for harnessing diffusion models in 3D-HPE, and demonstrate its ability to refine standard supervised 3D-HPE. We also show how diffusion models lead to more robust estimations in the face of occlusions, and improve the time-coherence and the sagittal symmetry of predictions. Using the Human\,3.6M dataset, we illustrate the effectiveness of our approach and its superiority over existing models, even under adverse situations where the occlusion patterns in training do not match those in inference. Our findings indicate that while standalone diffusion models provide commendable performance, their accuracy is even better in combination with supervised models, opening exciting new avenues for 3D-HPE research.