A Wasserstein GAN for Joint Learning of Inpainting and Spatial Optimisation

TL;DR

This paper introduces a novel Wasserstein GAN that jointly learns inpainting and mask optimization, improving image restoration quality and efficiency for applications like image compression.

Contribution

It presents the first GAN framework for simultaneous inpainting and spatial mask optimization, outperforming existing methods in quality and speed.

Findings

Significant visual quality improvements over traditional models

Faster inpainting and mask optimization process

Outperforms current spatial optimization networks

Abstract

Image inpainting is a restoration method that reconstructs missing image parts. However, a carefully selected mask of known pixels that yield a high quality inpainting can also act as a sparse image representation. This challenging spatial optimisation problem is essential for practical applications such as compression. So far, it has been almost exclusively adressed by model-based approaches. First attempts with neural networks seem promising, but are tailored towards specific inpainting operators or require postprocessing. To address this issue, we propose the first generative adversarial network (GAN) for spatial inpainting data optimisation. In contrast to previous approaches, it allows joint training of an inpainting generator and a corresponding mask optimisation network. With a Wasserstein distance, we ensure that our inpainting results accurately reflect the statistics of natural images. This yields significant improvements in visual quality and speed over conventional stochastic models. It also outperforms current spatial optimisation networks.