Semantic Latent Decomposition with Normalizing Flows for Face Editing

TL;DR

SDFlow introduces a novel framework using normalizing flows for semantic decomposition in latent space, enabling more precise and disentangled face editing compared to existing methods.

Contribution

The paper proposes SDFlow, a new approach that decomposes latent codes into semantic and irrelevant variables using continuous normalizing flows for improved face editing.

Findings

Outperforms state-of-the-art face editing methods

Provides more precise and disentangled attribute manipulation

Demonstrates effectiveness through qualitative and quantitative experiments

Abstract

Navigating in the latent space of StyleGAN has shown effectiveness for face editing. However, the resulting methods usually encounter challenges in complicated navigation due to the entanglement among different attributes in the latent space. To address this issue, this paper proposes a novel framework, termed SDFlow, with a semantic decomposition in original latent space using continuous conditional normalizing flows. Specifically, SDFlow decomposes the original latent code into different irrelevant variables by jointly optimizing two components: (i) a semantic encoder to estimate semantic variables from input faces and (ii) a flow-based transformation module to map the latent code into a semantic-irrelevant variable in Gaussian distribution, conditioned on the learned semantic variables. To eliminate the entanglement between variables, we employ a disentangled learning strategy under a mutual information framework, thereby providing precise manipulation controls. Experimental results demonstrate that SDFlow outperforms existing state-of-the-art face editing methods both qualitatively and quantitatively. The source code is made available at https://github.com/phil329/SDFlow.