Freestyle 3D-Aware Portrait Synthesis Based on Compositional Generative Priors

TL;DR

This paper introduces a novel text-driven 3D portrait synthesis framework that efficiently generates diverse, high-quality, and style-specific 3D portraits with view consistency, surpassing existing methods in speed and flexibility.

Contribution

The proposed method combines generative priors and a 3D latent feature generator to enable quick, out-of-distribution style portrait synthesis from text prompts.

Findings

Capable of generating high-quality stylized 3D portraits in minutes.

Outperforms state-of-the-art methods in style diversity and view consistency.

Efficiently constructs stylized 3D representations from text prompts.

Abstract

Efficiently generating a freestyle 3D portrait with high quality and 3D-consistency is a promising yet challenging task. The portrait styles generated by most existing methods are usually restricted by their 3D generators, which are learned in specific facial datasets, such as FFHQ. To get the diverse 3D portraits, one can build a large-scale multi-style database to retrain a 3D-aware generator, or use a off-the-shelf tool to do the style translation. However, the former is time-consuming due to data collection and training process, the latter may destroy the multi-view consistency. To tackle this problem, we propose a novel text-driven 3D-aware portrait synthesis framework that can generate out-of-distribution portrait styles. Specifically, for a given portrait style prompt, we first composite two generative priors, a 3D-aware GAN generator and a text-guided image editor, to quickly construct a few-shot stylized portrait set. Then we map the special style domain of this set to our proposed 3D latent feature generator and obtain a 3D representation containing the given style information. Finally we use a pre-trained 3D renderer to generate view-consistent stylized portraits from the 3D representation. Extensive experimental results show that our method is capable of synthesizing high-quality 3D portraits with specified styles in a few minutes, outperforming the state-of-the-art.