A Step to Decouple Optimization in 3DGS

TL;DR

This paper analyzes and improves the optimization process in 3D Gaussian Splatting by decoupling complex coupling components, leading to a more efficient and effective training method.

Contribution

It introduces a novel decoupling approach to the optimization in 3DGS, resulting in a re-designed optimizer called AdamW-GS that enhances efficiency and effectiveness.

Findings

Decoupling optimization components improves training efficiency.

Re-designed optimizer achieves better representation quality.

Empirical analysis validates the effectiveness of the new approach.

Abstract

3D Gaussian Splatting (3DGS) has emerged as a powerful technique for real-time novel view synthesis. As an explicit representation optimized through gradient propagation among primitives, optimization widely accepted in deep neural networks (DNNs) is actually adopted in 3DGS, such as synchronous weight updating and Adam with the adaptive gradient. However, considering the physical significance and specific design in 3DGS, there are two overlooked details in the optimization of 3DGS: (i) update step coupling, which induces optimizer state rescaling and costly attribute updates outside the viewpoints, and (ii) gradient coupling in the moment, which may lead to under- or over-effective regularization. Nevertheless, such a complex coupling is under-explored. After revisiting the optimization of 3DGS, we take a step to decouple it and recompose the process into: Sparse Adam, Re-State Regularization and Decoupled Attribute Regularization. Taking a large number of experiments under the 3DGS and 3DGS-MCMC frameworks, our work provides a deeper understanding of these components. Finally, based on the empirical analysis, we re-design the optimization and propose AdamW-GS by re-coupling the beneficial components, under which better optimization efficiency and representation effectiveness are achieved simultaneously.