LP-3DGS: Learning to Prune 3D Gaussian Splatting

TL;DR

This paper introduces LP-3DGS, a method that automatically learns the optimal pruning ratio for 3D Gaussian Splatting, reducing memory usage while maintaining high rendering quality through a differentiable masking approach.

Contribution

We propose a trainable binary mask with Gumbel-Sigmoid to automatically optimize pruning in 3D Gaussian Splatting, eliminating the need for manual hyperparameter tuning.

Findings

LP-3DGS reduces memory usage without quality loss.

The Gumbel-Sigmoid approach improves differentiability and training stability.

Experiments show consistent efficiency and quality balance.

Abstract

Recently, 3D Gaussian Splatting (3DGS) has become one of the mainstream methodologies for novel view synthesis (NVS) due to its high quality and fast rendering speed. However, as a point-based scene representation, 3DGS potentially generates a large number of Gaussians to fit the scene, leading to high memory usage. Improvements that have been proposed require either an empirical and preset pruning ratio or importance score threshold to prune the point cloud. Such hyperparamter requires multiple rounds of training to optimize and achieve the maximum pruning ratio, while maintaining the rendering quality for each scene. In this work, we propose learning-to-prune 3DGS (LP-3DGS), where a trainable binary mask is applied to the importance score that can find optimal pruning ratio automatically. Instead of using the traditional straight-through estimator (STE) method to approximate the binary mask gradient, we redesign the masking function to leverage the Gumbel-Sigmoid method, making it differentiable and compatible with the existing training process of 3DGS. Extensive experiments have shown that LP-3DGS consistently produces a good balance that is both efficient and high quality.