FaceLift: Learning Generalizable Single Image 3D Face Reconstruction from Synthetic Heads

TL;DR

FaceLift introduces a synthetic-data-trained, view-consistent 3D face reconstruction method from a single image, achieving high-quality, generalizable results with a novel multi-view generation and domain adaptation approach.

Contribution

The paper presents a new pipeline combining multi-view latent diffusion and transformer-based reconstruction trained solely on synthetic data for robust real-world 3D face reconstruction.

Findings

Outperforms state-of-the-art methods in identity preservation.

Achieves high detail recovery and rendering quality.

Demonstrates strong generalization to real-world images.

Abstract

We present FaceLift, a novel feed-forward approach for generalizable high-quality 360-degree 3D head reconstruction from a single image. Our pipeline first employs a multi-view latent diffusion model to generate consistent side and back views from a single facial input, which then feeds into a transformer-based reconstructor that produces a comprehensive 3D Gaussian splats representation. Previous methods for monocular 3D face reconstruction often lack full view coverage or view consistency due to insufficient multi-view supervision. We address this by creating a high-quality synthetic head dataset that enables consistent supervision across viewpoints. To bridge the domain gap between synthetic training data and real-world images, we propose a simple yet effective technique that ensures the view generation process maintains fidelity to the input by learning to reconstruct the input image alongside the view generation. Despite being trained exclusively on synthetic data, our method demonstrates remarkable generalization to real-world images. Through extensive qualitative and quantitative evaluations, we show that FaceLift outperforms state-of-the-art 3D face reconstruction methods on identity preservation, detail recovery, and rendering quality.