# Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting

## TL;DR

This paper introduces PEN-Net, a deep generative model with a pyramid-context encoder and multi-scale decoder, achieving high-quality image inpainting by ensuring both visual and semantic coherence.

## Contribution

The paper proposes a novel pyramid-context encoder within a U-Net framework for improved semantic and visual plausibility in image inpainting.

## Key findings

- PEN-Net outperforms existing methods on various datasets.
- The pyramid attention transfer enhances semantic consistency.
- Multi-scale supervision accelerates training and improves realism.

## Abstract

High-quality image inpainting requires filling missing regions in a damaged image with plausible content. Existing works either fill the regions by copying image patches or generating semantically-coherent patches from region context, while neglect the fact that both visual and semantic plausibility are highly-demanded. In this paper, we propose a Pyramid-context ENcoder Network (PEN-Net) for image inpainting by deep generative models. The PEN-Net is built upon a U-Net structure, which can restore an image by encoding contextual semantics from full resolution input, and decoding the learned semantic features back into images. Specifically, we propose a pyramid-context encoder, which progressively learns region affinity by attention from a high-level semantic feature map and transfers the learned attention to the previous low-level feature map. As the missing content can be filled by attention transfer from deep to shallow in a pyramid fashion, both visual and semantic coherence for image inpainting can be ensured. We further propose a multi-scale decoder with deeply-supervised pyramid losses and an adversarial loss. Such a design not only results in fast convergence in training, but more realistic results in testing. Extensive experiments on various datasets show the superior performance of the proposed network