RLGS: Reinforcement Learning-Based Adaptive Hyperparameter Tuning for Gaussian Splatting

TL;DR

RLGS is a reinforcement learning framework that adaptively tunes hyperparameters in 3D Gaussian Splatting, improving rendering quality and reducing manual effort across various models and datasets.

Contribution

It introduces a plug-and-play, model-agnostic RL-based hyperparameter tuning method for 3D Gaussian Splatting, enhancing performance without architectural changes.

Findings

Improves Taming-3DGS by 0.7dB PSNR on TNT dataset

Generalizes across multiple 3DGS variants

Enhances robustness across diverse datasets

Abstract

Hyperparameter tuning in 3D Gaussian Splatting (3DGS) is a labor-intensive and expert-driven process, often resulting in inconsistent reconstructions and suboptimal results. We propose RLGS, a plug-and-play reinforcement learning framework for adaptive hyperparameter tuning in 3DGS through lightweight policy modules, dynamically adjusting critical hyperparameters such as learning rates and densification thresholds. The framework is model-agnostic and seamlessly integrates into existing 3DGS pipelines without architectural modifications. We demonstrate its generalization ability across multiple state-of-the-art 3DGS variants, including Taming-3DGS and 3DGS-MCMC, and validate its robustness across diverse datasets. RLGS consistently enhances rendering quality. For example, it improves Taming-3DGS by 0.7dB PSNR on the Tanks and Temple (TNT) dataset, under a fixed Gaussian budget, and continues to yield gains even when baseline performance saturates. Our results suggest that RLGS provides an effective and general solution for automating hyperparameter tuning in 3DGS training, bridging a gap in applying reinforcement learning to 3DGS.