Large-Scale High-Quality 3D Gaussian Head Reconstruction from Multi-View Captures

TL;DR

HeadsUp is a scalable, high-quality 3D head reconstruction method from multi-view images that generalizes well and supports downstream applications like identity generation and expression animation.

Contribution

It introduces an efficient encoder-decoder architecture that decouples the number of 3D Gaussians from input views, enabling training on large-scale datasets with high-resolution inputs.

Findings

Achieves state-of-the-art reconstruction quality.

Generalizes to unseen identities without test-time optimization.

Effectively scales with dataset size and model capacity.

Abstract

We propose HeadsUp, a scalable feed-forward method for reconstructing high-quality 3D Gaussian heads from large-scale multi-camera setups. Our method employs an efficient encoder-decoder architecture that compresses input views into a compact latent representation. This latent representation is then decoded into a set of UV-parameterized 3D Gaussians anchored to a neutral head template. This UV representation decouples the number of 3D Gaussians from the number and resolution of input images, enabling training with many high-resolution input views. We train and evaluate our model on an internal dataset with more than 10,000 subjects, which is an order of magnitude larger than existing multi-view human head datasets. HeadsUp achieves state-of-the-art reconstruction quality and generalizes to novel identities without test-time optimization. We extensively analyze the scaling behavior of our model across identities, views, and model capacity, revealing practical insights for quality-compute trade-offs. Finally, we highlight the strength of our latent space by showcasing two downstream applications: generating novel 3D identities and animating the 3D heads with expression blendshapes.