One Shot 3D Photography
Johannes Kopf, Kevin Matzen, Suhib Alsisan, Ocean Quigley, Francis Ge,, Yangming Chong, Josh Patterson, Jan-Michael Frahm, Shu Wu, Matthew Yu,, Peizhao Zhang, Zijian He, Peter Vajda, Ayush Saraf, Michael Cohen

TL;DR
This paper introduces a mobile-optimized system for creating and viewing 3D photos from a single shot, using novel depth estimation and inpainting techniques to enable interactive parallax on various devices.
Contribution
The work presents a complete end-to-end pipeline for mobile 3D photo creation, featuring a new monocular depth network and inpainting method optimized for low latency and memory efficiency.
Findings
Competitive depth estimation accuracy with lower latency.
Fast processing time of a few seconds on mobile devices.
Effective rendering on low-end devices and over poor networks.
Abstract
3D photography is a new medium that allows viewers to more fully experience a captured moment. In this work, we refer to a 3D photo as one that displays parallax induced by moving the viewpoint (as opposed to a stereo pair with a fixed viewpoint). 3D photos are static in time, like traditional photos, but are displayed with interactive parallax on mobile or desktop screens, as well as on Virtual Reality devices, where viewing it also includes stereo. We present an end-to-end system for creating and viewing 3D photos, and the algorithmic and design choices therein. Our 3D photos are captured in a single shot and processed directly on a mobile device. The method starts by estimating depth from the 2D input image using a new monocular depth estimation network that is optimized for mobile devices. It performs competitively to the state-of-the-art, but has lower latency and peak memory…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Image Processing Techniques and Applications · Optical measurement and interference techniques
