BLADE: Single-view Body Mesh Learning through Accurate Depth Estimation

TL;DR

BLADE introduces a novel method for accurate perspective parameter estimation from a single image, significantly improving 3D human mesh recovery and 2D alignment, especially for close-range images, by avoiding heuristic assumptions.

Contribution

It presents a new approach to estimate perspective parameters directly from a single image, enhancing the accuracy of human mesh recovery in challenging close-range scenarios.

Findings

Achieves state-of-the-art 3D pose estimation accuracy.

Successfully recovers projection parameters without heuristics.

Performs well on standard benchmarks and real-world images.

Abstract

Single-image human mesh recovery is a challenging task due to the ill-posed nature of simultaneous body shape, pose, and camera estimation. Existing estimators work well on images taken from afar, but they break down as the person moves close to the camera. Moreover, current methods fail to achieve both accurate 3D pose and 2D alignment at the same time. Error is mainly introduced by inaccurate perspective projection heuristically derived from orthographic parameters. To resolve this long-standing challenge, we present our method BLADE which accurately recovers perspective parameters from a single image without heuristic assumptions. We start from the inverse relationship between perspective distortion and the person's Z-translation Tz, and we show that Tz can be reliably estimated from the image. We then discuss the important role of Tz for accurate human mesh recovery estimated from close-range images. Finally, we show that, once Tz and the 3D human mesh are estimated, one can accurately recover the focal length and full 3D translation. Extensive experiments on standard benchmarks and real-world close-range images show that our method is the first to accurately recover projection parameters from a single image, and consequently attain state-of-the-art accuracy on 3D pose estimation and 2D alignment for a wide range of images. https://research.nvidia.com/labs/amri/projects/blade/