Generative Approach for Probabilistic Human Mesh Recovery using Diffusion Models

TL;DR

This paper introduces Diff-HMR, a generative diffusion-based framework for 3D human mesh recovery from 2D images, effectively capturing the task's inherent ambiguity and producing diverse plausible results.

Contribution

It proposes a novel diffusion-based generative model for human mesh recovery, enabling probabilistic and diverse outputs unlike traditional deterministic methods.

Findings

Effectively models the ambiguity of human mesh recovery

Generates diverse plausible 3D meshes from a single image

Outperforms existing methods in probabilistic accuracy

Abstract

This work focuses on the problem of reconstructing a 3D human body mesh from a given 2D image. Despite the inherent ambiguity of the task of human mesh recovery, most existing works have adopted a method of regressing a single output. In contrast, we propose a generative approach framework, called "Diffusion-based Human Mesh Recovery (Diff-HMR)" that takes advantage of the denoising diffusion process to account for multiple plausible outcomes. During the training phase, the SMPL parameters are diffused from ground-truth parameters to random distribution, and Diff-HMR learns the reverse process of this diffusion. In the inference phase, the model progressively refines the given random SMPL parameters into the corresponding parameters that align with the input image. Diff-HMR, being a generative approach, is capable of generating diverse results for the same input image as the input noise varies. We conduct validation experiments, and the results demonstrate that the proposed framework effectively models the inherent ambiguity of the task of human mesh recovery in a probabilistic manner. The code is available at https://github.com/hanbyel0105/Diff-HMR