# The Missing Data Encoder: Cross-Channel Image Completion\\with Hide-And-Seek Adversarial Network

## TL;DR

This paper introduces the Missing Data Encoder (MDE), a deep adversarial network for image completion tasks like inpainting, extrapolation, and colorization, emphasizing a novel hide-and-seek loss for improved semantic understanding.

## Contribution

The paper proposes MDE with a new hide-and-seek adversarial loss and demonstrates its effectiveness in various image completion scenarios and unsupervised learning.

## Key findings

- MDE effectively performs image inpainting, extrapolation, and colorization.
- Training with random channel-independent fragments improves semantic capture.
- Models show strong results on multiple datasets for image completion and face occlusion.

## Abstract

Image completion is the problem of generating whole images from fragments only. It encompasses inpainting (generating a patch given its surrounding), reverse inpainting/extrapolation (generating the periphery given the central patch) as well as colorization (generating one or several channels given other ones). In this paper, we employ a deep network to perform image completion, with adversarial training as well as perceptual and completion losses, and call it the ``missing data encoder'' (MDE). We consider several configurations based on how the seed fragments are chosen. We show that training MDE for ``random extrapolation and colorization'' (MDE-REC), i.e. using random channel-independent fragments, allows a better capture of the image semantics and geometry. MDE training makes use of a novel ``hide-and-seek'' adversarial loss, where the discriminator seeks the original non-masked regions, while the generator tries to hide them. We validate our models both qualitatively and quantitatively on several datasets, showing their interest for image completion, unsupervised representation learning as well as face occlusion handling.