Image Inpainting with Learnable Feature Imputation

TL;DR

This paper introduces a novel, efficient feature imputation method for image inpainting that reduces artifacts and parameters, along with a revised gradient penalty and a new GAN architecture, achieving superior image quality.

Contribution

The paper proposes a minimal-parameter feature imputation technique for convolutional inpainting, a revised gradient penalty, and a new GAN architecture trained solely with adversarial loss.

Findings

Improved image quality on FDF dataset with the proposed methods.

Superior performance compared to state-of-the-art on CelebA-HQ and Places2.

Reduced artifacts and parameter count in inpainting models.

Abstract

A regular convolution layer applying a filter in the same way over known and unknown areas causes visual artifacts in the inpainted image. Several studies address this issue with feature re-normalization on the output of the convolution. However, these models use a significant amount of learnable parameters for feature re-normalization, or assume a binary representation of the certainty of an output. We propose (layer-wise) feature imputation of the missing input values to a convolution. In contrast to learned feature re-normalization, our method is efficient and introduces a minimal number of parameters. Furthermore, we propose a revised gradient penalty for image inpainting, and a novel GAN architecture trained exclusively on adversarial loss. Our quantitative evaluation on the FDF dataset reflects that our revised gradient penalty and alternative convolution improves generated image quality significantly. We present comparisons on CelebA-HQ and Places2 to current state-of-the-art to validate our model.