Real-Time Neural Light Field on Mobile Devices
Junli Cao, Huan Wang, Pavlo Chemerys, Vladislav Shakhrai, Ju Hu, Yun, Fu, Denys Makoviichuk, Sergey Tulyakov, Jian Ren

TL;DR
This paper introduces a novel, efficient neural rendering network that enables real-time, high-quality neural light field rendering on mobile devices, significantly reducing storage and latency compared to previous methods.
Contribution
The work presents a new mobile-friendly neural network architecture for neural light field rendering that achieves real-time performance and high image quality on resource-constrained devices.
Findings
Runs in 18.04ms on iPhone 13 for 1008x756 images
Reduces storage by 15-24 times compared to MobileNeRF
Achieves comparable or better image quality than NeRF and MobileNeRF
Abstract
Recent efforts in Neural Rendering Fields (NeRF) have shown impressive results on novel view synthesis by utilizing implicit neural representation to represent 3D scenes. Due to the process of volumetric rendering, the inference speed for NeRF is extremely slow, limiting the application scenarios of utilizing NeRF on resource-constrained hardware, such as mobile devices. Many works have been conducted to reduce the latency of running NeRF models. However, most of them still require high-end GPU for acceleration or extra storage memory, which is all unavailable on mobile devices. Another emerging direction utilizes the neural light field (NeLF) for speedup, as only one forward pass is performed on a ray to predict the pixel color. Nevertheless, to reach a similar rendering quality as NeRF, the network in NeLF is designed with intensive computation, which is not mobile-friendly. In this…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Computer Graphics and Visualization Techniques · Remote Sensing and LiDAR Applications
MethodsSPEED: Separable Pyramidal Pooling EncodEr-Decoder for Real-Time Monocular Depth Estimation on Low-Resource Settings
