PRoGS: Progressive Rendering of Gaussian Splats

TL;DR

This paper introduces PRoGS, a progressive rendering method for 3D Gaussian Splatting scenes that enables quick visualization of visible content without full scene loading, improving streaming and on-device rendering efficiency.

Contribution

It proposes a novel progressive rendering approach that prioritizes scene components, allowing faster visualization and better bandwidth optimization when combined with existing compression techniques.

Findings

Significant quality improvements over existing methods

Effective prioritization of scene splats for early visualization

Compatibility with compression techniques enhances streaming efficiency

Abstract

Over the past year, 3D Gaussian Splatting (3DGS) has received significant attention for its ability to represent 3D scenes in a perceptually accurate manner. However, it can require a substantial amount of storage since each splat's individual data must be stored. While compression techniques offer a potential solution by reducing the memory footprint, they still necessitate retrieving the entire scene before any part of it can be rendered. In this work, we introduce a novel approach for progressively rendering such scenes, aiming to display visible content that closely approximates the final scene as early as possible without loading the entire scene into memory. This approach benefits both on-device rendering applications limited by memory constraints and streaming applications where minimal bandwidth usage is preferred. To achieve this, we approximate the contribution of each Gaussian to the final scene and construct an order of prioritization on their inclusion in the rendering process. Additionally, we demonstrate that our approach can be combined with existing compression methods to progressively render (and stream) 3DGS scenes, optimizing bandwidth usage by focusing on the most important splats within a scene. Overall, our work establishes a foundation for making remotely hosted 3DGS content more quickly accessible to end-users in over-the-top consumption scenarios, with our results showing significant improvements in quality across all metrics compared to existing methods.