On Scaling Up 3D Gaussian Splatting Training

TL;DR

This paper introduces Grendel, a distributed system that enables scalable 3D Gaussian Splatting training across multiple GPUs, significantly improving reconstruction quality and handling large-scale scenes efficiently.

Contribution

We propose Grendel, a novel distributed framework for 3D Gaussian Splatting that supports multi-GPU training with sparse communication and dynamic load balancing.

Findings

Enhanced rendering quality with larger Gaussian models.

Achieved higher PSNR on large-scale scenes using multiple GPUs.

Demonstrated effective hyperparameter scaling strategies.

Abstract

3D Gaussian Splatting (3DGS) is increasingly popular for 3D reconstruction due to its superior visual quality and rendering speed. However, 3DGS training currently occurs on a single GPU, limiting its ability to handle high-resolution and large-scale 3D reconstruction tasks due to memory constraints. We introduce Grendel, a distributed system designed to partition 3DGS parameters and parallelize computation across multiple GPUs. As each Gaussian affects a small, dynamic subset of rendered pixels, Grendel employs sparse all-to-all communication to transfer the necessary Gaussians to pixel partitions and performs dynamic load balancing. Unlike existing 3DGS systems that train using one camera view image at a time, Grendel supports batched training with multiple views. We explore various optimization hyperparameter scaling strategies and find that a simple sqrt(batch size) scaling rule is highly effective. Evaluations using large-scale, high-resolution scenes show that Grendel enhances rendering quality by scaling up 3DGS parameters across multiple GPUs. On the Rubble dataset, we achieve a test PSNR of 27.28 by distributing 40.4 million Gaussians across 16 GPUs, compared to a PSNR of 26.28 using 11.2 million Gaussians on a single GPU. Grendel is an open-source project available at: https://github.com/nyu-systems/Grendel-GS