3DGS$^2$-TR: Scalable Second-Order Trust-Region Method for 3D Gaussian Splatting

TL;DR

This paper introduces 3DGS$^2$-TR, a scalable second-order optimizer for 3D Gaussian Splatting that approximates curvature efficiently, enabling faster scene training with less memory and improved reconstruction quality.

Contribution

It proposes a fully matrix-free second-order optimization method using Hessian diagonal approximation and a trust-region technique for stable, efficient 3D scene training.

Findings

Achieves better reconstruction quality with 50% fewer iterations.

Uses less than 1GB GPU memory, scalable to large scenes.

Outperforms ADAM in training speed and quality.

Abstract

We propose 3DGS$^2$-TR,a second-order optimizer for accelerating the scene training problem in 3D Gaussian Splatting (3DGS). Unlike existing second-order approaches that rely on explicit or dense curvature representations, such as 3DGS-LM (H\"ollein et al., 2025) or 3DGS2 (Lan et al., 2025), our method approximates curvature using only the diagonal of the Hessian matrix, efficiently via Hutchinson's method. Our approach is fully matrix-free and has the same complexity as ADAM (Kingma, 2024), $O(n)$ in both computation and memory costs. To ensure stable optimization in the presence of strong nonlinearity in the 3DGS rasterization process, we introduce a parameter-wise trust-region technique based on the squared Hellinger distance, regularizing updates to Gaussian parameters. Under identical parameter initialization and without densification, 3DGS$^2$-TR is able to achieve better reconstruction quality on standard datasets, using 50% fewer training iterations compared to ADAM, while incurring less than 1GB of peak GPU memory overhead (17% more than ADAM and 85% less than 3DGS-LM), enabling scalability to very large scenes and potentially to distributed training settings.