Deep Curvilinear Editing: Commutative and Nonlinear Image Manipulation for Pretrained Deep Generative Model

TL;DR

This paper introduces DeCurvEd, a novel nonlinear, commutative semantic editing method for GANs that improves image attribute disentanglement and editing quality by ensuring order-independent attribute manipulation.

Contribution

DeCurvEd is the first method to establish nonlinear, commuting vector fields in latent space for semantic image editing, enabling consistent multi-attribute manipulation.

Findings

DeCurvEd achieves higher editing quality than previous methods.

The method ensures attribute editing is order-independent.

Experimental results show improved attribute disentanglement.

Abstract

Semantic editing of images is the fundamental goal of computer vision. Although deep learning methods, such as generative adversarial networks (GANs), are capable of producing high-quality images, they often do not have an inherent way of editing generated images semantically. Recent studies have investigated a way of manipulating the latent variable to determine the images to be generated. However, methods that assume linear semantic arithmetic have certain limitations in terms of the quality of image editing, whereas methods that discover nonlinear semantic pathways provide non-commutative editing, which is inconsistent when applied in different orders. This study proposes a novel method called deep curvilinear editing (DeCurvEd) to determine semantic commuting vector fields on the latent space. We theoretically demonstrate that owing to commutativity, the editing of multiple attributes depends only on the quantities and not on the order. Furthermore, we experimentally demonstrate that compared to previous methods, the nonlinear and commutative nature of DeCurvEd facilitates the disentanglement of image attributes and provides higher-quality editing.