R-MNet: A Perceptual Adversarial Network for Image Inpainting

TL;DR

This paper introduces R-MNet, a novel perceptual adversarial network utilizing reverse masking and a feature space loss for improved high-resolution image inpainting, producing more realistic and coherent results.

Contribution

The paper proposes a new reverse mask operator and a feature space loss within a Wasserstein GAN framework for enhanced image inpainting quality.

Findings

Achieves realistic and coherent inpainting results.

Generalizes well to high-resolution images.

Outperforms state-of-the-art methods in realism.

Abstract

Facial image inpainting is a problem that is widely studied, and in recent years the introduction of Generative Adversarial Networks, has led to improvements in the field. Unfortunately some issues persists, in particular when blending the missing pixels with the visible ones. We address the problem by proposing a Wasserstein GAN combined with a new reverse mask operator, namely Reverse Masking Network (R-MNet), a perceptual adversarial network for image inpainting. The reverse mask operator transfers the reverse masked image to the end of the encoder-decoder network leaving only valid pixels to be inpainted. Additionally, we propose a new loss function computed in feature space to target only valid pixels combined with adversarial training. These then capture data distributions and generate images similar to those in the training data with achieved realism (realistic and coherent) on the output images. We evaluate our method on publicly available dataset, and compare with state-of-the-art methods. We show that our method is able to generalize to high-resolution inpainting task, and further show more realistic outputs that are plausible to the human visual system when compared with the state-of-the-art methods.