Image Generation with Self Pixel-wise Normalization

TL;DR

This paper introduces Self Pixel-wise Normalization (SPN), a novel method that enhances GAN-based image generation by adaptively normalizing features without needing external mask images, improving quality metrics across datasets.

Contribution

The paper proposes SPN, a mask-free pixel-wise normalization technique that boosts image generation performance in GANs by deriving self-latent masks from features.

Findings

SPN improves FID and IS scores significantly.

SPN effectively captures foreground objects in generated images.

SPN is easily integrated into existing models.

Abstract

Region-adaptive normalization (RAN) methods have been widely used in the generative adversarial network (GAN)-based image-to-image translation technique. However, since these approaches need a mask image to infer the pixel-wise affine transformation parameters, they cannot be applied to the general image generation models having no paired mask images. To resolve this problem, this paper presents a novel normalization method, called self pixel-wise normalization (SPN), which effectively boosts the generative performance by performing the pixel-adaptive affine transformation without the mask image. In our method, the transforming parameters are derived from a self-latent mask that divides the feature map into the foreground and background regions. The visualization of the self-latent masks shows that SPN effectively captures a single object to be generated as the foreground. Since the proposed method produces the self-latent mask without external data, it is easily applicable in the existing generative models. Extensive experiments on various datasets reveal that the proposed method significantly improves the performance of image generation technique in terms of Frechet inception distance (FID) and Inception score (IS).