Training and Tuning Generative Neural Radiance Fields for Attribute-Conditional 3D-Aware Face Generation

TL;DR

This paper introduces a conditional GNeRF model that enhances attribute control and disentanglement in 3D-aware face generation by integrating attribute labels and a novel training method, enabling high-quality, view-consistent edits.

Contribution

It proposes a new conditional GNeRF framework with a TRIOT training method that improves attribute editing and disentanglement without retraining from scratch.

Findings

Enhanced attribute control and view consistency in 3D-aware face generation.

Effective attribute editing with high fidelity and minimal unintended changes.

Model outperforms existing methods in attribute disentanglement and editing precision.

Abstract

Generative Neural Radiance Fields (GNeRF)-based 3D-aware GANs have showcased remarkable prowess in crafting high-fidelity images while upholding robust 3D consistency, particularly face generation. However, specific existing models prioritize view consistency over disentanglement, leading to constrained semantic or attribute control during the generation process. While many methods have explored incorporating semantic masks or leveraging 3D Morphable Models (3DMM) priors to imbue models with semantic control, these methods often demand training from scratch, entailing significant computational overhead. In this paper, we propose a novel approach: a conditional GNeRF model that integrates specific attribute labels as input, thus amplifying the controllability and disentanglement capabilities of 3D-aware generative models. Our approach builds upon a pre-trained 3D-aware face model, and we introduce a Training as Init and Optimizing for Tuning (TRIOT) method to train a conditional normalized flow module to enable the facial attribute editing, then optimize the latent vector to improve attribute-editing precision further. Our extensive experiments substantiate the efficacy of our model, showcasing its ability to generate high-quality edits with enhanced view consistency while safeguarding non-target regions. The code for our model is publicly available at https://github.com/zhangqianhui/TT-GNeRF.