Image Inpainting using Block-wise Procedural Training with Annealed Adversarial Counterpart

TL;DR

This paper introduces a novel deep learning framework for image inpainting that employs block-wise procedural training and adversarial loss annealing to produce high-quality, realistic results with fewer artifacts and better continuity.

Contribution

The authors propose a new training scheme and loss strategy for deep generative inpainting models, improving stability, quality, and practical usability over existing methods.

Findings

Outperforms existing inpainting methods in quality and realism

Reduces artifacts and discontinuities near filled regions

Enables interactive guided inpainting applications

Abstract

Recent advances in deep generative models have shown promising potential in image inpanting, which refers to the task of predicting missing pixel values of an incomplete image using the known context. However, existing methods can be slow or generate unsatisfying results with easily detectable flaws. In addition, there is often perceivable discontinuity near the holes and require further post-processing to blend the results. We present a new approach to address the difficulty of training a very deep generative model to synthesize high-quality photo-realistic inpainting. Our model uses conditional generative adversarial networks (conditional GANs) as the backbone, and we introduce a novel block-wise procedural training scheme to stabilize the training while we increase the network depth. We also propose a new strategy called adversarial loss annealing to reduce the artifacts. We further describe several losses specifically designed for inpainting and show their effectiveness. Extensive experiments and user-study show that our approach outperforms existing methods in several tasks such as inpainting, face completion and image harmonization. Finally, we show our framework can be easily used as a tool for interactive guided inpainting, demonstrating its practical value to solve common real-world challenges.