Single Image, Any Face: Generalisable 3D Face Generation

TL;DR

This paper introduces Gen3D-Face, a novel diffusion-based model that generates photorealistic 3D human faces from a single unconstrained image, achieving high generalization and multi-view consistency without requiring ground-truth 3D data.

Contribution

The paper presents the first unified framework for single-image 3D face generation that works across domains, using a multi-view diffusion approach and subject-specific mesh estimation.

Findings

Outperforms previous methods in out-of-domain scenarios

Achieves top results in in-domain competitions

Demonstrates high multi-view consistency and realism

Abstract

The creation of 3D human face avatars from a single unconstrained image is a fundamental task that underlies numerous real-world vision and graphics applications. Despite the significant progress made in generative models, existing methods are either less suited in design for human faces or fail to generalise from the restrictive training domain to unconstrained facial images. To address these limitations, we propose a novel model, Gen3D-Face, which generates 3D human faces with unconstrained single image input within a multi-view consistent diffusion framework. Given a specific input image, our model first produces multi-view images, followed by neural surface construction. To incorporate face geometry information while preserving generalisation to in-the-wild inputs, we estimate a subject-specific mesh directly from the input image, enabling training and evaluation without ground-truth 3D supervision. Importantly, we introduce a multi-view joint generation scheme to enhance the appearance consistency among different views. To the best of our knowledge, this is the first attempt and benchmark for creating photorealistic 3D human face avatars from single images for generic human subject across domains. Extensive experiments demonstrate the efficacy and superiority of our method over previous alternatives for out-of-domain single image 3D face generation and the top ranking competition for the in-domain setting.