Exploring the Effectiveness of Mask-Guided Feature Modulation as a Mechanism for Localized Style Editing of Real Images

TL;DR

This paper introduces SemanticStyle Autoencoder (SSAE), a novel method using mask-guided feature modulation in the latent space for efficient, localized, and photorealistic style editing of real images, avoiding expensive inversion and supervision.

Contribution

It proposes SSAE, a deep autoencoder leveraging semantic mask-guided latent manipulation for localized style editing, reducing computational cost and human supervision.

Findings

Effective localized style editing demonstrated

Outperforms existing methods in quality and efficiency

Provides both qualitative and quantitative validation

Abstract

The success of Deep Generative Models at high-resolution image generation has led to their extensive utilization for style editing of real images. Most existing methods work on the principle of inverting real images onto their latent space, followed by determining controllable directions. Both inversion of real images and determination of controllable latent directions are computationally expensive operations. Moreover, the determination of controllable latent directions requires additional human supervision. This work aims to explore the efficacy of mask-guided feature modulation in the latent space of a Deep Generative Model as a solution to these bottlenecks. To this end, we present the SemanticStyle Autoencoder (SSAE), a deep Generative Autoencoder model that leverages semantic mask-guided latent space manipulation for highly localized photorealistic style editing of real images. We present qualitative and quantitative results for the same and their analysis. This work shall serve as a guiding primer for future work.