Perceive-Sample-Compress: Towards Real-Time 3D Gaussian Splatting

TL;DR

This paper introduces a novel framework for 3D Gaussian Splatting that enhances scene perception, compresses data efficiently, and maintains real-time, high-quality rendering in complex environments.

Contribution

It proposes a perception-based refinement algorithm, a pyramid sampling scheme, and a Gaussian compression method to improve efficiency and quality in 3D Gaussian Splatting.

Findings

Significantly reduces memory usage while preserving visual fidelity.

Achieves real-time rendering with high-quality novel view synthesis.

Demonstrates effectiveness on complex scenes with large-scale environments.

Abstract

Recent advances in 3D Gaussian Splatting (3DGS) have demonstrated remarkable capabilities in real-time and photorealistic novel view synthesis. However, traditional 3DGS representations often struggle with large-scale scene management and efficient storage, particularly when dealing with complex environments or limited computational resources. To address these limitations, we introduce a novel perceive-sample-compress framework for 3D Gaussian Splatting. Specifically, we propose a scene perception compensation algorithm that intelligently refines Gaussian parameters at each level. This algorithm intelligently prioritizes visual importance for higher fidelity rendering in critical areas, while optimizing resource usage and improving overall visible quality. Furthermore, we propose a pyramid sampling representation to manage Gaussian primitives across hierarchical levels. Finally, to facilitate efficient storage of proposed hierarchical pyramid representations, we develop a Generalized Gaussian Mixed model compression algorithm to achieve significant compression ratios without sacrificing visual fidelity. The extensive experiments demonstrate that our method significantly improves memory efficiency and high visual quality while maintaining real-time rendering speed.