Unsupervised Adversarial Image Inpainting

TL;DR

This paper introduces an unsupervised image inpainting method using a conditional GAN that models a distribution of plausible reconstructions without requiring paired training data, demonstrating effectiveness across various datasets.

Contribution

It presents a novel unsupervised inpainting approach leveraging a conditional GAN with explicit stochastic dependencies, enabling sampling of multiple plausible reconstructions from incomplete observations.

Findings

Achieves comparable results to supervised models on multiple datasets

Effectively models a distribution over reconstructions without paired data

Demonstrates versatility across different image types and masks

Abstract

We consider inpainting in an unsupervised setting where there is neither access to paired nor unpaired training data. The only available information is provided by the uncomplete observations and the inpainting process statistics. In this context, an observation should give rise to several plausible reconstructions which amounts at learning a distribution over the space of reconstructed images. We model the reconstruction process by using a conditional GAN with constraints on the stochastic component that introduce an explicit dependency between this component and the generated output. This allows us sampling from the latent component in order to generate a distribution of images associated to an observation. We demonstrate the capacity of our model on several image datasets: faces (CelebA), food images (Recipe-1M) and bedrooms (LSUN Bedrooms) with different types of imputation masks. The approach yields comparable performance to model variants trained with additional supervision.