Structure-guided Image Outpainting

TL;DR

This paper introduces a GAN-based image outpainting method that uses structural priors and semantic embedding to improve realism and structural consistency in extrapolated images.

Contribution

It proposes a novel deep learning framework incorporating structural priors and semantic embedding loss for improved image outpainting quality.

Findings

Enhanced structural realism in outpainted images

Improved perceptual quality with semantic embedding loss

Effective multi-phase adversarial training scheme

Abstract

Deep learning techniques have made considerable progress in image inpainting, restoration, and reconstruction in the last few years. Image outpainting, also known as image extrapolation, lacks attention and practical approaches to be fulfilled, owing to difficulties caused by large-scale area loss and less legitimate neighboring information. These difficulties have made outpainted images handled by most of the existing models unrealistic to human eyes and spatially inconsistent. When upsampling through deconvolution to generate fake content, the naive generation methods may lead to results lacking high-frequency details and structural authenticity. Therefore, as our novelties to handle image outpainting problems, we introduce structural prior as a condition to optimize the generation quality and a new semantic embedding term to enhance perceptual sanity. we propose a deep learning method based on Generative Adversarial Network (GAN) and condition edges as structural prior in order to assist the generation. We use a multi-phase adversarial training scheme that comprises edge inference training, contents inpainting training, and joint training. The newly added semantic embedding loss is proved effective in practice.