FlexGS: Train Once, Deploy Everywhere with Many-in-One Flexible 3D Gaussian Splatting

TL;DR

FlexGS introduces an elastic inference approach for 3D Gaussian splatting that allows adaptable model sizes for different device constraints without fine-tuning, maintaining high rendering quality.

Contribution

We propose a novel elastic inference method for 3D Gaussian splatting that dynamically selects and transforms Gaussians based on target memory constraints, eliminating the need for fine-tuning.

Findings

Effective model size adaptation without fine-tuning.

Maintains high rendering quality across different device constraints.

Demonstrated on multiple 3D scene datasets.

Abstract

3D Gaussian splatting (3DGS) has enabled various applications in 3D scene representation and novel view synthesis due to its efficient rendering capabilities. However, 3DGS demands relatively significant GPU memory, limiting its use on devices with restricted computational resources. Previous approaches have focused on pruning less important Gaussians, effectively compressing 3DGS but often requiring a fine-tuning stage and lacking adaptability for the specific memory needs of different devices. In this work, we present an elastic inference method for 3DGS. Given an input for the desired model size, our method selects and transforms a subset of Gaussians, achieving substantial rendering performance without additional fine-tuning. We introduce a tiny learnable module that controls Gaussian selection based on the input percentage, along with a transformation module that adjusts the selected Gaussians to complement the performance of the reduced model. Comprehensive experiments on ZipNeRF, MipNeRF and Tanks\&Temples scenes demonstrate the effectiveness of our approach. Code is available at https://flexgs.github.io.