Wavelet Prior Attention Learning in Axial Inpainting Network

TL;DR

This paper introduces WAIN, a novel image inpainting model that leverages wavelet prior attention and axial transformers to improve semantic coherence and reduce artifacts, achieving state-of-the-art results.

Contribution

The paper proposes Wavelet Prior Attention and stacked axial transformers within an inpainting network to enhance feature aggregation and semantic coherence with fewer prior parameters.

Findings

Achieves state-of-the-art performance on Celeba-HQ and Places2 datasets.

Effectively reduces texture artifacts in inpainted images.

Improves semantic coherence in reconstructed images.

Abstract

Image inpainting is the task of filling masked or unknown regions of an image with visually realistic contents, which has been remarkably improved by Deep Neural Networks (DNNs) recently. Essentially, as an inverse problem, the inpainting has the underlying challenges of reconstructing semantically coherent results without texture artifacts. Many previous efforts have been made via exploiting attention mechanisms and prior knowledge, such as edges and semantic segmentation. However, these works are still limited in practice by an avalanche of learnable prior parameters and prohibitive computational burden. To this end, we propose a novel model -- Wavelet prior attention learning in Axial Inpainting Network (WAIN), whose generator contains the encoder, decoder, as well as two key components of Wavelet image Prior Attention (WPA) and stacked multi-layer Axial-Transformers (ATs). Particularly, the WPA guides the high-level feature aggregation in the multi-scale frequency domain, alleviating the textual artifacts. Stacked ATs employ unmasked clues to help model reasonable features along with low-level features of horizontal and vertical axes, improving the semantic coherence. Extensive quantitative and qualitative experiments on Celeba-HQ and Places2 datasets are conducted to validate that our WAIN can achieve state-of-the-art performance over the competitors. The codes and models will be released.