Region-aware Attention for Image Inpainting

TL;DR

This paper introduces a region-aware attention module and a learnable region dictionary to improve image inpainting, reducing blurriness and enhancing semantic plausibility by better modeling pixel correlations across samples.

Contribution

The proposed region-aware attention and learnable region dictionary are novel techniques that address correlation misguidance and redundancy in image inpainting.

Findings

Outperforms existing methods on CelebA, Places2, and Paris StreetView datasets.

Generates semantically plausible and detailed inpainted images.

Effectively reduces blurriness caused by incorrect pixel correlations.

Abstract

Recent attention-based image inpainting methods have made inspiring progress by modeling long-range dependencies within a single image. However, they tend to generate blurry contents since the correlation between each pixel pairs is always misled by ill-predicted features in holes. To handle this problem, we propose a novel region-aware attention (RA) module. By avoiding the directly calculating corralation between each pixel pair in a single samples and considering the correlation between different samples, the misleading of invalid information in holes can be avoided. Meanwhile, a learnable region dictionary (LRD) is introduced to store important information in the entire dataset, which not only simplifies correlation modeling, but also avoids information redundancy. By applying RA in our architecture, our methodscan generate semantically plausible results with realistic details. Extensive experiments on CelebA, Places2 and Paris StreetView datasets validate the superiority of our method compared with existing methods.