AttDiff-GAN: A Hybrid Diffusion-GAN Framework for Facial Attribute Editing

TL;DR

AttDiff-GAN introduces a hybrid diffusion-GAN framework for facial attribute editing that improves accuracy and attribute preservation by combining feature-level adversarial learning with diffusion guidance.

Contribution

The paper proposes a novel hybrid framework that decouples attribute editing from image synthesis, integrating facial priors and semantic relationships for enhanced editing precision.

Findings

Achieves more accurate facial attribute editing on CelebA-HQ.

Better preservation of non-target attributes compared to state-of-the-art methods.

Demonstrates superior qualitative and quantitative results in experiments.

Abstract

Facial attribute editing aims to modify target attributes while preserving attribute-irrelevant content and overall image fidelity. Existing GAN-based methods provide favorable controllability, but often suffer from weak alignment between style codes and attribute semantics. Diffusion-based methods can synthesize highly realistic images; however, their editing precision is limited by the entanglement of semantic directions among different attributes. In this paper, we propose AttDiff-GAN, a hybrid framework that combines GAN-based attribute manipulation with diffusion-based image generation. A key challenge in such integration lies in the inconsistency between one-step adversarial learning and multi-step diffusion denoising, which makes effective optimization difficult. To address this issue, we decouple attribute editing from image synthesis by introducing a feature-level adversarial learning scheme to learn explicit attribute manipulation, and then using the manipulated features to guide the diffusion process for image generation, while also removing the reliance on semantic direction-based editing. Moreover, we enhance style-attribute alignment by introducing PriorMapper, which incorporates facial priors into style generation, and RefineExtractor, which captures global semantic relationships through a Transformer for more precise style extraction. Experimental results on CelebA-HQ show that the proposed method achieves more accurate facial attribute editing and better preservation of non-target attributes than state-of-the-art methods in both qualitative and quantitative evaluations.