3D Photography using Context-aware Layered Depth Inpainting
Meng-Li Shih, Shih-Yang Su, Johannes Kopf, Jia-Bin Huang

TL;DR
This paper introduces a learning-based method to generate 3D photos from a single RGB-D image by inpainting occluded regions with context-aware color and depth, enabling realistic view synthesis with fewer artifacts.
Contribution
It presents a novel layered depth inpainting approach that effectively hallucinates occluded regions for 3D photo creation from a single image.
Findings
Fewer artifacts compared to state-of-the-art methods.
Effective in diverse everyday scenes.
Enables efficient rendering with motion parallax.
Abstract
We propose a method for converting a single RGB-D input image into a 3D photo - a multi-layer representation for novel view synthesis that contains hallucinated color and depth structures in regions occluded in the original view. We use a Layered Depth Image with explicit pixel connectivity as underlying representation, and present a learning-based inpainting model that synthesizes new local color-and-depth content into the occluded region in a spatial context-aware manner. The resulting 3D photos can be efficiently rendered with motion parallax using standard graphics engines. We validate the effectiveness of our method on a wide range of challenging everyday scenes and show fewer artifacts compared with the state of the arts.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
3D Photography Using Context-Aware Layered Depth Inpainting· youtube
Taxonomy
TopicsAdvanced Vision and Imaging · Computer Graphics and Visualization Techniques · Generative Adversarial Networks and Image Synthesis
