Diffusion-based Pose Refinement and Muti-hypothesis Generation for 3D Human Pose Estimaiton

TL;DR

This paper introduces DRPose, a diffusion-based framework that refines deterministic 3D human pose estimates and generates multiple plausible hypotheses, achieving state-of-the-art results on key benchmarks.

Contribution

The paper proposes a novel diffusion-based refinement framework with a scalable graph transformer and pose refinement module for improved 3D human pose estimation.

Findings

Achieves state-of-the-art performance on Human3.6M and MPI-INF-3DHP datasets.

Effectively refines deterministic pose estimates through reverse diffusion.

Generates accurate multi-hypothesis predictions for 3D human pose.

Abstract

Previous probabilistic models for 3D Human Pose Estimation (3DHPE) aimed to enhance pose accuracy by generating multiple hypotheses. However, most of the hypotheses generated deviate substantially from the true pose. Compared to deterministic models, the excessive uncertainty in probabilistic models leads to weaker performance in single-hypothesis prediction. To address these two challenges, we propose a diffusion-based refinement framework called DRPose, which refines the output of deterministic models by reverse diffusion and achieves more suitable multi-hypothesis prediction for the current pose benchmark by multi-step refinement with multiple noises. To this end, we propose a Scalable Graph Convolution Transformer (SGCT) and a Pose Refinement Module (PRM) for denoising and refining. Extensive experiments on Human3.6M and MPI-INF-3DHP datasets demonstrate that our method achieves state-of-the-art performance on both single and multi-hypothesis 3DHPE. Code is available at https://github.com/KHB1698/DRPose.