Towards Real-Time Neural Volumetric Rendering on Mobile Devices: A Measurement Study

TL;DR

This study systematically evaluates the performance factors affecting real-time neural radiance field rendering on mobile devices, highlighting key system controls and their impact on efficiency and quality.

Contribution

It provides the first comprehensive measurement-based analysis of NeRF rendering system performance from a system perspective, identifying critical control knobs.

Findings

Mesh granularity significantly improves performance.

Quantization has minimal impact on system efficiency.

Hardware and network conditions influence optimal control settings.

Abstract

Neural Radiance Fields (NeRF) is an emerging technique to synthesize 3D objects from 2D images with a wide range of potential applications. However, rendering existing NeRF models is extremely computation intensive, making it challenging to support real-time interaction on mobile devices. In this paper, we take the first initiative to examine the state-of-the-art real-time NeRF rendering technique from a system perspective. We first define the entire working pipeline of the NeRF serving system. We then identify possible control knobs that are critical to the system from the communication, computation, and visual performance perspective. Furthermore, an extensive measurement study is conducted to reveal the effects of these control knobs on system performance. Our measurement results reveal that different control knobs contribute differently towards improving the system performance, with the mesh granularity being the most effective knob and the quantization being the least effective knob. In addition, diverse hardware device settings and network conditions have to be considered to fully unleash the benefit of operating under the appropriate knobs