Attentive Normalization for Conditional Image Generation

TL;DR

This paper introduces attentive normalization, a method that improves conditional image generation by modeling long-range dependencies more efficiently than self-attention, leading to better consistency in generated images.

Contribution

It proposes attentive normalization as an extension to instance normalization, capturing semantic long-range dependencies without high computational costs.

Findings

Enhanced image consistency across distant regions

Efficient modeling of long-range dependencies

Improved results in class-conditional image generation

Abstract

Traditional convolution-based generative adversarial networks synthesize images based on hierarchical local operations, where long-range dependency relation is implicitly modeled with a Markov chain. It is still not sufficient for categories with complicated structures. In this paper, we characterize long-range dependence with attentive normalization (AN), which is an extension to traditional instance normalization. Specifically, the input feature map is softly divided into several regions based on its internal semantic similarity, which are respectively normalized. It enhances consistency between distant regions with semantic correspondence. Compared with self-attention GAN, our attentive normalization does not need to measure the correlation of all locations, and thus can be directly applied to large-size feature maps without much computational burden. Extensive experiments on class-conditional image generation and semantic inpainting verify the efficacy of our proposed module.