SparseSplat: Towards Applicable Feed-Forward 3D Gaussian Splatting with Pixel-Unaligned Prediction

TL;DR

SparseSplat introduces an adaptive, entropy-based 3D Gaussian Splatting method that produces compact, scene-aware maps, significantly reducing the number of Gaussians needed while maintaining high rendering quality.

Contribution

It is the first feed-forward 3DGS model that adaptively adjusts Gaussian density based on scene structure, enabling highly compact maps and improved efficiency.

Findings

Achieves state-of-the-art rendering quality with only 22% of Gaussians.

Maintains reasonable quality with just 1.5% of Gaussians.

Uses entropy-based sampling and a specialized point cloud network for encoding.

Abstract

Recent progress in feed-forward 3D Gaussian Splatting (3DGS) has notably improved rendering quality. However, the spatially uniform and highly redundant 3DGS map generated by previous feed-forward 3DGS methods limits their integration into downstream reconstruction tasks. We propose SparseSplat, the first feed-forward 3DGS model that adaptively adjusts Gaussian density according to scene structure and information richness of local regions, yielding highly compact 3DGS maps. To achieve this, we propose entropy-based probabilistic sampling, generating large, sparse Gaussians in textureless areas and assigning small, dense Gaussians to regions with rich information. Additionally, we designed a specialized point cloud network that efficiently encodes local context and decodes it into 3DGS attributes, addressing the receptive field mismatch between the general 3DGS optimization pipeline and feed-forward models. Extensive experimental results demonstrate that SparseSplat can achieve state-of-the-art rendering quality with only 22% of the Gaussians and maintain reasonable rendering quality with only 1.5% of the Gaussians. Project page: https://victkk.github.io/SparseSplat-page/.