Flexible Geometric Guidance for Probabilistic Human Pose Estimation with Diffusion Models

TL;DR

This paper introduces a diffusion model-based framework for probabilistic 3D human pose estimation from 2D images, enabling sampling of multiple plausible poses and demonstrating state-of-the-art results without requiring paired 2D-3D training data.

Contribution

It proposes a novel guidance framework using diffusion models for pose estimation, allowing flexible sampling and generalization to new tasks without training dedicated models.

Findings

Achieves state-of-the-art performance without paired 2D-3D data.

Demonstrates strong generalization on unseen datasets.

Enables pose generation and completion without additional training.

Abstract

3D human pose estimation from 2D images is a challenging problem due to depth ambiguity and occlusion. Because of these challenges the task is underdetermined, where there exists multiple -- possibly infinite -- poses that are plausible given the image. Despite this, many prior works assume the existence of a deterministic mapping and estimate a single pose given an image. Furthermore, methods based on machine learning require a large amount of paired 2D-3D data to train and suffer from generalization issues to unseen scenarios. To address both of these issues, we propose a framework for pose estimation using diffusion models, which enables sampling from a probability distribution over plausible poses which are consistent with a 2D image. Our approach falls under the guidance framework for conditional generation, and guides samples from an unconditional diffusion model, trained only on 3D data, using the gradients of the heatmaps from a 2D keypoint detector. We evaluate our method on the Human 3.6M dataset under best-of-$m$ multiple hypothesis evaluation, showing state-of-the-art performance among methods which do not require paired 2D-3D data for training. We additionally evaluate the generalization ability using the MPI-INF-3DHP and 3DPW datasets and demonstrate competitive performance. Finally, we demonstrate the flexibility of our framework by using it for novel tasks including pose generation and pose completion, without the need to train bespoke conditional models. We make code available at https://github.com/fsnelgar/diffusion_pose .