SiENet: Siamese Expansion Network for Image Extrapolation

TL;DR

SiENet is a novel two-stage siamese adversarial network designed for image outpainting, effectively predicting unknown border content by leveraging adaptive border-sensitive convolutions and prior knowledge, outperforming existing methods.

Contribution

Introduces a two-stage siamese adversarial model with adaptive border-sensitive convolutions for improved image outpainting.

Findings

Outperforms state-of-the-art methods on four datasets.

Produces more realistic and coherent extrapolated images.

Effectively models long-range feature distribution for better prediction.

Abstract

Different from image inpainting, image outpainting has relative less context in the image center to capture and more content at the image border to predict. Therefore, classical encoder-decoder pipeline of existing methods may not predict the outstretched unknown content perfectly. In this paper, a novel two-stage siamese adversarial model for image extrapolation, named Siamese Expansion Network (SiENet) is proposed. In two stages, a novel border sensitive convolution named adaptive filling convolution is designed for allowing encoder to predict the unknown content, alleviating the burden of decoder. Besides, to introduce prior knowledge to network and reinforce the inferring ability of encoder, siamese adversarial mechanism is designed to enable our network to model the distribution of covered long range feature for that of uncovered image feature. The results on four datasets has demonstrated that our method outperforms existing state-of-the-arts and could produce realistic results.