Pixel-wise Dense Detector for Image Inpainting

TL;DR

This paper introduces a pixel-wise dense detector for image inpainting that improves artifact localization and automatically balances loss functions, leading to superior inpainting results.

Contribution

It proposes a novel detection-based framework with pixel-wise artifact localization and automatic loss balancing for improved image inpainting.

Findings

Outperforms existing methods on multiple datasets

Effectively localizes artifacts at pixel level

Automatically balances adversarial and reconstruction losses

Abstract

Recent GAN-based image inpainting approaches adopt an average strategy to discriminate the generated image and output a scalar, which inevitably lose the position information of visual artifacts. Moreover, the adversarial loss and reconstruction loss (e.g., l1 loss) are combined with tradeoff weights, which are also difficult to tune. In this paper, we propose a novel detection-based generative framework for image inpainting, which adopts the min-max strategy in an adversarial process. The generator follows an encoder-decoder architecture to fill the missing regions, and the detector using weakly supervised learning localizes the position of artifacts in a pixel-wise manner. Such position information makes the generator pay attention to artifacts and further enhance them. More importantly, we explicitly insert the output of the detector into the reconstruction loss with a weighting criterion, which balances the weight of the adversarial loss and reconstruction loss automatically rather than manual operation. Experiments on multiple public datasets show the superior performance of the proposed framework. The source code is available at https://github.com/Evergrow/GDN_Inpainting.