SG-Splatting: Accelerating 3D Gaussian Splatting with Spherical Gaussians

TL;DR

SG-Splatting introduces a novel spherical Gaussian-based color representation for 3D Gaussian Splatting, significantly enhancing rendering speed and quality while reducing memory usage for real-time view synthesis.

Contribution

The paper proposes a new spherical Gaussian-based color representation and an optimized scene organization strategy to improve 3D Gaussian Splatting's efficiency and visual fidelity.

Findings

Reduced memory footprint compared to spherical harmonics.

Accelerated rendering speed in novel view synthesis.

Enhanced visual quality through mixed color representations.

Abstract

3D Gaussian Splatting is emerging as a state-of-the-art technique in novel view synthesis, recognized for its impressive balance between visual quality, speed, and rendering efficiency. However, reliance on third-degree spherical harmonics for color representation introduces significant storage demands and computational overhead, resulting in a large memory footprint and slower rendering speed. We introduce SG-Splatting with Spherical Gaussians based color representation, a novel approach to enhance rendering speed and quality in novel view synthesis. Our method first represents view-dependent color using Spherical Gaussians, instead of three degree spherical harmonics, which largely reduces the number of parameters used for color representation, and significantly accelerates the rendering process. We then develop an efficient strategy for organizing multiple Spherical Gaussians, optimizing their arrangement to achieve a balanced and accurate scene representation. To further improve rendering quality, we propose a mixed representation that combines Spherical Gaussians with low-degree spherical harmonics, capturing both high- and low-frequency color information effectively. SG-Splatting also has plug-and-play capability, allowing it to be easily integrated into existing systems. This approach improves computational efficiency and overall visual fidelity, making it a practical solution for real-time applications.