Deep Stacked Networks with Residual Polishing for Image Inpainting

TL;DR

This paper introduces a two-stage deep learning framework for image inpainting that first fills missing regions and then refines the result to remove artifacts, leading to improved visual quality.

Contribution

It proposes a novel stacked CNN architecture with separate inpainting and artifact removal stages, adaptable to various inpainting methods.

Findings

Significant visual quality improvement in inpainted images

Quantitative metrics show enhanced accuracy

Framework effectively reduces artifacts and noise

Abstract

Deep neural networks have shown promising results in image inpainting even if the missing area is relatively large. However, most of the existing inpainting networks introduce undesired artifacts and noise to the repaired regions. To solve this problem, we present a novel framework which consists of two stacked convolutional neural networks that inpaint the image and remove the artifacts, respectively. The first network considers the global structure of the damaged image and coarsely fills the blank area. Then the second network modifies the repaired image to cancel the noise introduced by the first network. The proposed framework splits the problem into two distinct partitions that can be optimized separately, therefore it can be applied to any inpainting algorithm by changing the first network. Second stage in our framework which aims at polishing the inpainted images can be treated as a denoising problem where a wide range of algorithms can be employed. Our results demonstrate that the proposed framework achieves significant improvement on both visual and quantitative evaluations.