Efficient Neural Light Fields (ENeLF) for Mobile Devices

TL;DR

This paper introduces ENeLF, a neural light field model optimized for mobile devices, combining a novel architecture with aggressive pruning to enable efficient, real-time novel view synthesis with minimal performance loss.

Contribution

It presents a mobile-friendly neural light field model using a new architecture and pruning techniques, reducing computational cost and model size for practical mobile deployment.

Findings

Achieves real-time rendering on mobile devices

Reduces model size and latency significantly

Maintains comparable rendering quality with slight performance decrease

Abstract

Novel view synthesis (NVS) is a challenge in computer vision and graphics, focusing on generating realistic images of a scene from unobserved camera poses, given a limited set of authentic input images. Neural radiance fields (NeRF) achieved impressive results in rendering quality by utilizing volumetric rendering. However, NeRF and its variants are unsuitable for mobile devices due to the high computational cost of volumetric rendering. Emerging research in neural light fields (NeLF) eliminates the need for volumetric rendering by directly learning a mapping from ray representation to pixel color. NeLF has demonstrated its capability to achieve results similar to NeRF but requires a more extensive, computationally intensive network that is not mobile-friendly. Unlike existing works, this research builds upon the novel network architecture introduced by MobileR2L and aggressively applies a compression technique (channel-wise structure pruning) to produce a model that runs efficiently on mobile devices with lower latency and smaller sizes, with a slight decrease in performance.