Generator Pyramid for High-Resolution Image Inpainting

TL;DR

This paper introduces PyramidFill, a multi-scale GAN framework that improves high-resolution image inpainting by separately handling content completion at low resolution and texture synthesis at high resolution, achieving superior results.

Contribution

The novel PyramidFill framework explicitly disentangles content and texture synthesis across resolutions using a pyramid of specialized GANs, advancing high-resolution inpainting techniques.

Findings

Outperforms state-of-the-art methods on multiple datasets.

Effectively handles large holes in high-resolution images.

Demonstrates superior quality in natural scenery inpainting.

Abstract

Inpainting high-resolution images with large holes challenges existing deep learning based image inpainting methods. We present a novel framework -- PyramidFill for high-resolution image inpainting task, which explicitly disentangles content completion and texture synthesis. PyramidFill attempts to complete the content of unknown regions in a lower-resolution image, and synthesis the textures of unknown regions in a higher-resolution image, progressively. Thus, our model consists of a pyramid of fully convolutional GANs, wherein the content GAN is responsible for completing contents in the lowest-resolution masked image, and each texture GAN is responsible for synthesizing textures in a higher-resolution image. Since completing contents and synthesising textures demand different abilities from generators, we customize different architectures for the content GAN and texture GAN. Experiments on multiple datasets including CelebA-HQ, Places2 and a new natural scenery dataset (NSHQ) with different resolutions demonstrate that PyramidFill generates higher-quality inpainting results than the state-of-the-art methods. To better assess high-resolution image inpainting methods, we will release NSHQ, high-quality natural scenery images with high-resolution 1920$\times$1080.