ReFu: Refine and Fuse the Unobserved View for Detail-Preserving Single-Image 3D Human Reconstruction

TL;DR

ReFu is a novel method that refines and fuses unobserved view information to improve detail-preserving 3D human reconstruction from a single image, achieving state-of-the-art results.

Contribution

It introduces a coarse-to-fine refinement approach with occupancy supervision and backside view image generation for better 3D reconstruction.

Findings

Achieves state-of-the-art performance in 3D human reconstruction.

Enhances geometry and texture quality from unobserved views.

Effectively suppresses noise in projection images and meshes.

Abstract

Single-image 3D human reconstruction aims to reconstruct the 3D textured surface of the human body given a single image. While implicit function-based methods recently achieved reasonable reconstruction performance, they still bear limitations showing degraded quality in both surface geometry and texture from an unobserved view. In response, to generate a realistic textured surface, we propose ReFu, a coarse-to-fine approach that refines the projected backside view image and fuses the refined image to predict the final human body. To suppress the diffused occupancy that causes noise in projection images and reconstructed meshes, we propose to train occupancy probability by simultaneously utilizing 2D and 3D supervisions with occupancy-based volume rendering. We also introduce a refinement architecture that generates detail-preserving backside-view images with front-to-back warping. Extensive experiments demonstrate that our method achieves state-of-the-art performance in 3D human reconstruction from a single image, showing enhanced geometry and texture quality from an unobserved view.