Deep Multi Depth Panoramas for View Synthesis
Kai-En Lin, Zexiang Xu, Ben Mildenhall, Pratul P. Srinivasan, Yannick, Hold-Geoffroy, Stephen DiVerdi, Qi Sun, Kalyan Sunkavalli, and Ravi, Ramamoorthi

TL;DR
This paper introduces Multi Depth Panoramas (MDPs), a novel scene representation for view synthesis from 360-degree panoramas, capable of handling large translations and view-dependent effects more effectively than previous methods.
Contribution
The paper presents a deep learning approach to reconstruct MDPs from multi-camera panoramas, enabling high-quality view synthesis with a more compact scene representation.
Findings
MDPs outperform previous RGBD-based methods in view synthesis quality.
The approach effectively handles large translations and view-dependent effects.
Experiments show improved efficiency and realism in synthesized views.
Abstract
We propose a learning-based approach for novel view synthesis for multi-camera 360 panorama capture rigs. Previous work constructs RGBD panoramas from such data, allowing for view synthesis with small amounts of translation, but cannot handle the disocclusions and view-dependent effects that are caused by large translations. To address this issue, we present a novel scene representation - Multi Depth Panorama (MDP) - that consists of multiple RGBD panoramas that represent both scene geometry and appearance. We demonstrate a deep neural network-based method to reconstruct MDPs from multi-camera 360 images. MDPs are more compact than previous 3D scene representations and enable high-quality, efficient new view rendering. We demonstrate this via experiments on both synthetic and real data and comparisons with previous state-of-the-art methods spanning both…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Robotics and Sensor-Based Localization · Advanced Image and Video Retrieval Techniques
