Object-Driven Multi-Layer Scene Decomposition From a Single Image

TL;DR

This paper introduces an object-driven, adaptive layered depth image reconstruction method from a single RGB image, improving occlusion handling and scene completion for applications like 3D photography.

Contribution

It presents a novel adaptive, object-driven approach that predicts layered depth images with semantic encoding from a single image, handling occlusions more effectively than prior methods.

Findings

Accurately reconstructs occluded scene regions.

Enables applications like 3D photography and diminished reality.

Outperforms previous methods in scene completion quality.

Abstract

We present a method that tackles the challenge of predicting color and depth behind the visible content of an image. Our approach aims at building up a Layered Depth Image (LDI) from a single RGB input, which is an efficient representation that arranges the scene in layers, including originally occluded regions. Unlike previous work, we enable an adaptive scheme for the number of layers and incorporate semantic encoding for better hallucination of partly occluded objects. Additionally, our approach is object-driven, which especially boosts the accuracy for the occluded intermediate objects. The framework consists of two steps. First, we individually complete each object in terms of color and depth, while estimating the scene layout. Second, we rebuild the scene based on the regressed layers and enforce the recomposed image to resemble the structure of the original input. The learned representation enables various applications, such as 3D photography and diminished reality, all from a single RGB image.