NVGS: Neural Visibility for Occlusion Culling in 3D Gaussian Splatting

TL;DR

This paper introduces NVGS, a neural visibility method for occlusion culling in 3D Gaussian Splatting, improving rendering efficiency and image quality by discarding occluded primitives.

Contribution

It proposes a novel neural visibility function learned via a small MLP, integrated into a rasterizer for efficient occlusion culling in 3D Gaussian rendering.

Findings

Outperforms state-of-the-art in VRAM usage and image quality

Efficiently discards occluded primitives during rendering

Complementary to existing level-of-detail techniques

Abstract

3D Gaussian Splatting can exploit frustum culling and level-of-detail strategies to accelerate rendering of scenes containing a large number of primitives. However, the semi-transparent nature of Gaussians prevents the application of another highly effective technique: occlusion culling. We address this limitation by proposing a novel method to learn the viewpoint-dependent visibility function of all Gaussians in a trained model using a small, shared MLP across instances of an asset in a scene. By querying it for Gaussians within the viewing frustum prior to rasterization, our method can discard occluded primitives during rendering. Leveraging Tensor Cores for efficient computation, we integrate these neural queries directly into a novel instanced software rasterizer. Our approach outperforms the current state of the art for composed scenes in terms of VRAM usage and image quality, utilizing a combination of our instanced rasterizer and occlusion culling MLP, and exhibits complementary properties to existing LoD techniques.