SaiNet: Stereo aware inpainting behind objects with generative networks

TL;DR

SaiNet is a stereo-aware inpainting network that uses edge-guided UNet with partial convolutions and a disparity loss to achieve stereo-consistent inpainting behind objects, trained on realistic stereo masks.

Contribution

The paper introduces a novel stereo-aware inpainting method with a disparity loss and a training scheme based on realistic stereo masks, improving over prior random-mask approaches.

Findings

Achieves competitive results with state-of-the-art methods.

Enforces stereo consistency through a disparity loss.

Utilizes realistic stereo masks for supervised training.

Abstract

In this work, we present an end-to-end network for stereo-consistent image inpainting with the objective of inpainting large missing regions behind objects. The proposed model consists of an edge-guided UNet-like network using Partial Convolutions. We enforce multi-view stereo consistency by introducing a disparity loss. More importantly, we develop a training scheme where the model is learned from realistic stereo masks representing object occlusions, instead of the more common random masks. The technique is trained in a supervised way. Our evaluation shows competitive results compared to previous state-of-the-art techniques.