SemFaceEdit: Semantic Face Editing on Generative Radiance Manifolds

TL;DR

SemFaceEdit introduces a novel method for precise semantic face editing on generative radiance manifolds, enabling disentangled control over facial features while maintaining view consistency and computational efficiency.

Contribution

It proposes a new approach that generates semantic fields on radiance manifolds for localized, disentangled facial editing, improving over existing methods in control and detail preservation.

Findings

Enhanced semantic editing precision

Improved disentanglement of facial features

Superior performance in radiance field control

Abstract

Despite multiple view consistency offered by 3D-aware GAN techniques, the resulting images often lack the capacity for localized editing. In response, generative radiance manifolds emerge as an efficient approach for constrained point sampling within volumes, effectively reducing computational demands and enabling the learning of fine details. This work introduces SemFaceEdit, a novel method that streamlines the appearance and geometric editing process by generating semantic fields on generative radiance manifolds. Utilizing latent codes, our method effectively disentangles the geometry and appearance associated with different facial semantics within the generated image. In contrast to existing methods that can change the appearance of the entire radiance field, our method enables the precise editing of particular facial semantics while preserving the integrity of other regions. Our network comprises two key modules: the Geometry module, which generates semantic radiance and occupancy fields, and the Appearance module, which is responsible for predicting RGB radiance. We jointly train both modules in adversarial settings to learn semantic-aware geometry and appearance descriptors. The appearance descriptors are then conditioned on their respective semantic latent codes by the Appearance Module, facilitating disentanglement and enhanced control. Our experiments highlight SemFaceEdit's superior performance in semantic field-based editing, particularly in achieving improved radiance field disentanglement.