Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes

TL;DR

This paper presents a novel neural implicit surface representation using an octree-based feature volume that enables real-time rendering of high-fidelity 3D shapes with state-of-the-art quality and efficiency.

Contribution

It introduces an adaptive octree-based neural SDF with continuous LOD and a fast rendering algorithm, significantly improving speed and quality over prior methods.

Findings

Achieves 2-3 orders of magnitude faster rendering than previous neural SDF methods.

Maintains state-of-the-art shape reconstruction quality for complex geometries.

Enables real-time rendering of high-fidelity neural implicit surfaces.

Abstract

Neural signed distance functions (SDFs) are emerging as an effective representation for 3D shapes. State-of-the-art methods typically encode the SDF with a large, fixed-size neural network to approximate complex shapes with implicit surfaces. Rendering with these large networks is, however, computationally expensive since it requires many forward passes through the network for every pixel, making these representations impractical for real-time graphics. We introduce an efficient neural representation that, for the first time, enables real-time rendering of high-fidelity neural SDFs, while achieving state-of-the-art geometry reconstruction quality. We represent implicit surfaces using an octree-based feature volume which adaptively fits shapes with multiple discrete levels of detail (LODs), and enables continuous LOD with SDF interpolation. We further develop an efficient algorithm to directly render our novel neural SDF representation in real-time by querying only the necessary LODs with sparse octree traversal. We show that our representation is 2-3 orders of magnitude more efficient in terms of rendering speed compared to previous works. Furthermore, it produces state-of-the-art reconstruction quality for complex shapes under both 3D geometric and 2D image-space metrics.