High-Resolution Volumetric Reconstruction for Clothed Humans

TL;DR

This paper introduces a volumetric reconstruction method for clothed humans from few images, leveraging 3D convolutions and a coarse-to-fine strategy to achieve high accuracy and realistic rendering.

Contribution

It demonstrates that a well-designed volumetric approach with system optimizations outperforms recent implicit methods in clothed human reconstruction.

Findings

Reduces point-to-surface error by over 50% to about 2mm.

Achieves higher PSNR in rendered images compared to state-of-the-art.

Uses a coarse-to-fine strategy with voxel culling for efficiency.

Abstract

We present a novel method for reconstructing clothed humans from a sparse set of, e.g., 1 to 6 RGB images. Despite impressive results from recent works employing deep implicit representation, we revisit the volumetric approach and demonstrate that better performance can be achieved with proper system design. The volumetric representation offers significant advantages in leveraging 3D spatial context through 3D convolutions, and the notorious quantization error is largely negligible with a reasonably large yet affordable volume resolution, e.g., 512. To handle memory and computation costs, we propose a sophisticated coarse-to-fine strategy with voxel culling and subspace sparse convolution. Our method starts with a discretized visual hull to compute a coarse shape and then focuses on a narrow band nearby the coarse shape for refinement. Once the shape is reconstructed, we adopt an image-based rendering approach, which computes the colors of surface points by blending input images with learned weights. Extensive experimental results show that our method significantly reduces the mean point-to-surface (P2S) precision of state-of-the-art methods by more than 50% to achieve approximately 2mm accuracy with a 512 volume resolution. Additionally, images rendered from our textured model achieve a higher peak signal-to-noise ratio (PSNR) compared to state-of-the-art methods.