Virtual Memory for 3D Gaussian Splatting

TL;DR

This paper introduces a virtual memory-based method for efficiently rendering large 3D Gaussian Splatting scenes, enabling real-time visualization of complex environments with reduced memory usage and improved speed.

Contribution

It presents a novel approach combining virtual memory and texturing techniques to stream only necessary Gaussians for real-time rendering of large-scale scenes.

Findings

Enables real-time rendering of large 3D scenes on desktop and mobile devices.

Reduces memory usage by streaming only visible Gaussians.

Improves rendering speed with level of detail integration.

Abstract

3D Gaussian Splatting represents a breakthrough in the field of novel view synthesis. It establishes Gaussians as core rendering primitives for highly accurate real-world environment reconstruction. Recent advances have drastically increased the size of scenes that can be created. In this work, we present a method for rendering large and complex 3D Gaussian Splatting scenes using virtual memory. By leveraging well-established virtual memory and virtual texturing techniques, our approach efficiently identifies visible Gaussians and dynamically streams them to the GPU just in time for real-time rendering. Selecting only the necessary Gaussians for both storage and rendering results in reduced memory usage and effectively accelerates rendering, especially for highly complex scenes. Furthermore, we demonstrate how level of detail can be integrated into our proposed method to further enhance rendering speed for large-scale scenes. With an optimized implementation, we highlight key practical considerations and thoroughly evaluate the proposed technique and its impact on desktop and mobile devices.