Implicit Shape and Appearance Priors for Few-Shot Full Head Reconstruction

TL;DR

This paper introduces a probabilistic shape and appearance prior for coordinate-based neural representations, enabling fast and accurate few-shot 3D head reconstruction from minimal input images, outperforming previous methods in speed and quality.

Contribution

It proposes a novel probabilistic prior integrated into coordinate-based neural representations to improve few-shot 3D head reconstruction, reducing data requirements and computational time.

Findings

Achieves state-of-the-art geometry reconstruction quality.

Runs an order of magnitude faster than previous methods.

Successfully reconstructs 3D heads from as little as one image.

Abstract

Recent advancements in learning techniques that employ coordinate-based neural representations have yielded remarkable results in multi-view 3D reconstruction tasks. However, these approaches often require a substantial number of input views (typically several tens) and computationally intensive optimization procedures to achieve their effectiveness. In this paper, we address these limitations specifically for the problem of few-shot full 3D head reconstruction. We accomplish this by incorporating a probabilistic shape and appearance prior into coordinate-based representations, enabling faster convergence and improved generalization when working with only a few input images (even as low as a single image). During testing, we leverage this prior to guide the fitting process of a signed distance function using a differentiable renderer. By incorporating the statistical prior alongside parallelizable ray tracing and dynamic caching strategies, we achieve an efficient and accurate approach to few-shot full 3D head reconstruction. Moreover, we extend the H3DS dataset, which now comprises 60 high-resolution 3D full head scans and their corresponding posed images and masks, which we use for evaluation purposes. By leveraging this dataset, we demonstrate the remarkable capabilities of our approach in achieving state-of-the-art results in geometry reconstruction while being an order of magnitude faster than previous approaches.