Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstruction

TL;DR

Geo-PIFu introduces a novel implicit function-based approach for accurate 3D human mesh reconstruction from single images, leveraging a 3D U-Net to encode features and regularize shape, significantly improving detail and reducing distortion.

Contribution

The paper presents Geo-PIFu, a new method combining geometry-aware implicit functions and voxel features for superior single-view human reconstruction.

Findings

Achieves 42.7% lower Chamfer and Point-to-Surface distances.

Reduces normal estimation errors by 19.4%.

Outperforms previous methods on a large human mesh dataset.

Abstract

We propose Geo-PIFu, a method to recover a 3D mesh from a monocular color image of a clothed person. Our method is based on a deep implicit function-based representation to learn latent voxel features using a structure-aware 3D U-Net, to constrain the model in two ways: first, to resolve feature ambiguities in query point encoding, second, to serve as a coarse human shape proxy to regularize the high-resolution mesh and encourage global shape regularity. We show that, by both encoding query points and constraining global shape using latent voxel features, the reconstruction we obtain for clothed human meshes exhibits less shape distortion and improved surface details compared to competing methods. We evaluate Geo-PIFu on a recent human mesh public dataset that is $10 \times$ larger than the private commercial dataset used in PIFu and previous derivative work. On average, we exceed the state of the art by $42.7\%$ reduction in Chamfer and Point-to-Surface Distances, and $19.4\%$ reduction in normal estimation errors.