Distribution-Aligned Diffusion for Human Mesh Recovery

TL;DR

This paper introduces HMDiff, a diffusion-based framework for 3D human mesh recovery from a single image, incorporating a distribution alignment technique to improve accuracy and achieve state-of-the-art results.

Contribution

The paper proposes a novel diffusion-based approach with a distribution alignment technique for improved human mesh recovery from images.

Findings

Achieves state-of-the-art performance on multiple datasets.

Effectively incorporates prior distribution information into mesh recovery.

Demonstrates the effectiveness of diffusion models in 3D human mesh estimation.

Abstract

Recovering a 3D human mesh from a single RGB image is a challenging task due to depth ambiguity and self-occlusion, resulting in a high degree of uncertainty. Meanwhile, diffusion models have recently seen much success in generating high-quality outputs by progressively denoising noisy inputs. Inspired by their capability, we explore a diffusion-based approach for human mesh recovery, and propose a Human Mesh Diffusion (HMDiff) framework which frames mesh recovery as a reverse diffusion process. We also propose a Distribution Alignment Technique (DAT) that infuses prior distribution information into the mesh distribution diffusion process, and provides useful prior knowledge to facilitate the mesh recovery task. Our method achieves state-of-the-art performance on three widely used datasets. Project page: https://gongjia0208.github.io/HMDiff/.