A Reference-Based 3D Semantic-Aware Framework for Accurate Local Facial Attribute Editing

TL;DR

This paper presents a novel 3D-aware facial attribute editing framework that combines latent and reference-based methods, ensuring precise, realistic, and multi-view consistent modifications with improved realism and control.

Contribution

The proposed framework introduces a 3D GAN inversion and blending approach that enhances accuracy and realism in facial attribute editing from multiple viewpoints.

Findings

Outperforms existing methods in multi-view consistency

Achieves more realistic and precise attribute modifications

Effective in preserving image integrity during editing

Abstract

Facial attribute editing plays a crucial role in synthesizing realistic faces with specific characteristics while maintaining realistic appearances. Despite advancements, challenges persist in achieving precise, 3D-aware attribute modifications, which are crucial for consistent and accurate representations of faces from different angles. Current methods struggle with semantic entanglement and lack effective guidance for incorporating attributes while maintaining image integrity. To address these issues, we introduce a novel framework that merges the strengths of latent-based and reference-based editing methods. Our approach employs a 3D GAN inversion technique to embed attributes from the reference image into a tri-plane space, ensuring 3D consistency and realistic viewing from multiple perspectives. We utilize blending techniques and predicted semantic masks to locate precise edit regions, merging them with the contextual guidance from the reference image. A coarse-to-fine inpainting strategy is then applied to preserve the integrity of untargeted areas, significantly enhancing realism. Our evaluations demonstrate superior performance across diverse editing tasks, validating our framework's effectiveness in realistic and applicable facial attribute editing.