# Pluralistic Image Completion

## TL;DR

This paper introduces a probabilistically principled framework for pluralistic image completion, generating multiple diverse plausible solutions by combining reconstructive and generative paths supported by GANs and a novel attention layer.

## Contribution

It proposes a novel dual-path framework with attention mechanisms to produce diverse image completions, overcoming the limitations of single-solution methods.

## Key findings

- Generated higher-quality, diverse completions on multiple datasets.
- Outperformed existing methods in diversity and quality of results.
- Introduced a new attention layer for better appearance consistency.

## Abstract

Most image completion methods produce only one result for each masked input, although there may be many reasonable possibilities. In this paper, we present an approach for \textbf{pluralistic image completion} -- the task of generating multiple and diverse plausible solutions for image completion. A major challenge faced by learning-based approaches is that usually only one ground truth training instance per label. As such, sampling from conditional VAEs still leads to minimal diversity. To overcome this, we propose a novel and probabilistically principled framework with two parallel paths. One is a reconstructive path that utilizes the only one given ground truth to get prior distribution of missing parts and rebuild the original image from this distribution. The other is a generative path for which the conditional prior is coupled to the distribution obtained in the reconstructive path. Both are supported by GANs. We also introduce a new short+long term attention layer that exploits distant relations among decoder and encoder features, improving appearance consistency. When tested on datasets with buildings (Paris), faces (CelebA-HQ), and natural images (ImageNet), our method not only generated higher-quality completion results, but also with multiple and diverse plausible outputs.