Improve Deep Image Inpainting by Emphasizing the Complexity of Missing Regions

TL;DR

This paper introduces a simple, effective method that uses classical image complexity metrics to guide training batch selection, significantly improving deep image inpainting performance across multiple models and datasets.

Contribution

It proposes a knowledge-assisted index combining missingness complexity and forward loss to enhance deep inpainting training, integrating traditional image processing with deep learning.

Findings

Improved inpainting results on various datasets.

Enhanced performance across multiple deep inpainting models.

Method is easy to implement and plug into existing models.

Abstract

Deep image inpainting research mainly focuses on constructing various neural network architectures or imposing novel optimization objectives. However, on the one hand, building a state-of-the-art deep inpainting model is an extremely complex task, and on the other hand, the resulting performance gains are sometimes very limited. We believe that besides the frameworks of inpainting models, lightweight traditional image processing techniques, which are often overlooked, can actually be helpful to these deep models. In this paper, we enhance the deep image inpainting models with the help of classical image complexity metrics. A knowledge-assisted index composed of missingness complexity and forward loss is presented to guide the batch selection in the training procedure. This index helps find samples that are more conducive to optimization in each iteration and ultimately boost the overall inpainting performance. The proposed approach is simple and can be plugged into many deep inpainting models by changing only a few lines of code. We experimentally demonstrate the improvements for several recently developed image inpainting models on various datasets.