SteerNeRF: Accelerating NeRF Rendering via Smooth Viewpoint Trajectory

TL;DR

SteerNeRF introduces a method to accelerate NeRF rendering by exploiting smooth viewpoint trajectories, reducing computation while maintaining quality, enabling real-time 1080p rendering with low memory use.

Contribution

The paper proposes a novel approach that leverages smooth viewpoint changes to speed up NeRF rendering without high memory costs.

Findings

Achieves 30FPS at 1080P resolution.

Maintains competitive rendering quality.

Uses minimal additional memory.

Abstract

Neural Radiance Fields (NeRF) have demonstrated superior novel view synthesis performance but are slow at rendering. To speed up the volume rendering process, many acceleration methods have been proposed at the cost of large memory consumption. To push the frontier of the efficiency-memory trade-off, we explore a new perspective to accelerate NeRF rendering, leveraging a key fact that the viewpoint change is usually smooth and continuous in interactive viewpoint control. This allows us to leverage the information of preceding viewpoints to reduce the number of rendered pixels as well as the number of sampled points along the ray of the remaining pixels. In our pipeline, a low-resolution feature map is rendered first by volume rendering, then a lightweight 2D neural renderer is applied to generate the output image at target resolution leveraging the features of preceding and current frames. We show that the proposed method can achieve competitive rendering quality while reducing the rendering time with little memory overhead, enabling 30FPS at 1080P image resolution with a low memory footprint.