Surrogate Gradient Field for Latent Space Manipulation

TL;DR

This paper introduces a novel method using Surrogate Gradient Field for latent space manipulation in GANs, enabling complex attribute editing like keypoints and captions with improved disentanglement.

Contribution

It presents the first approach to manipulate GAN latent codes with multidimensional conditions, outperforming existing methods in disentanglement and versatility.

Findings

Outperforms state-of-the-art in disentanglement for facial attribute editing

Successfully manipulates complex properties like keypoints and captions

Introduces a new metric for evaluating disentanglement in manipulation methods

Abstract

Generative adversarial networks (GANs) can generate high-quality images from sampled latent codes. Recent works attempt to edit an image by manipulating its underlying latent code, but rarely go beyond the basic task of attribute adjustment. We propose the first method that enables manipulation with multidimensional condition such as keypoints and captions. Specifically, we design an algorithm that searches for a new latent code that satisfies the target condition based on the Surrogate Gradient Field (SGF) induced by an auxiliary mapping network. For quantitative comparison, we propose a metric to evaluate the disentanglement of manipulation methods. Thorough experimental analysis on the facial attribute adjustment task shows that our method outperforms state-of-the-art methods in disentanglement. We further apply our method to tasks of various condition modalities to demonstrate that our method can alter complex image properties such as keypoints and captions.