CheapNVS: Real-Time On-Device Narrow-Baseline Novel View Synthesis
Konstantinos Georgiadis, Mehmet Kerim Yucel, and Albert Saa-Garriga

TL;DR
CheapNVS is a fast, efficient, real-time on-device method for narrow-baseline single-view novel view synthesis that outperforms state-of-the-art approaches in speed, memory, and quality.
Contribution
It introduces a novel multi-stage encoder/decoder architecture that enables real-time, on-device NVS with significant performance improvements.
Findings
Outperforms state-of-the-art in quality and efficiency
Runs at over 30 FPS on a Samsung Tab 9+
Uses 6% less memory than previous methods
Abstract
Single-view novel view synthesis (NVS) is a notorious problem due to its ill-posed nature, and often requires large, computationally expensive approaches to produce tangible results. In this paper, we propose CheapNVS: a fully end-to-end approach for narrow baseline single-view NVS based on a novel, efficient multiple encoder/decoder design trained in a multi-stage fashion. CheapNVS first approximates the laborious 3D image warping with lightweight learnable modules that are conditioned on the camera pose embeddings of the target view, and then performs inpainting on the occluded regions in parallel to achieve significant performance gains. Once trained on a subset of Open Images dataset, CheapNVS outperforms the state-of-the-art despite being 10 times faster and consuming 6% less memory. Furthermore, CheapNVS runs comfortably in real-time on mobile devices, reaching over 30 FPS on a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsCCD and CMOS Imaging Sensors · Advanced Vision and Imaging · Image Processing Techniques and Applications
MethodsInpainting
