TRIM: Scalable 3D Gaussian Diffusion Inference with Temporal and Spatial Trimming

TL;DR

TRIM is a post-training method that accelerates 3D Gaussian diffusion inference by using temporal and spatial trimming strategies, reducing computation while maintaining high output quality.

Contribution

It introduces a lightweight selector model and instance mask denoising to improve inference efficiency and scalability of 3D Gaussian diffusion models.

Findings

Significant speedup in 3D diffusion inference

Maintained or improved generation quality

Enhanced scalability along sampling trajectories

Abstract

Recent advances in 3D Gaussian diffusion models suffer from time-intensive denoising and post-denoising processing due to the massive number of Gaussian primitives, resulting in slow generation and limited scalability along sampling trajectories. To improve the efficiency of 3D diffusion models, we propose $\textbf{TRIM}$ ($\textbf{T}$rajectory $\textbf{R}$eduction and $\textbf{I}$nstance $\textbf{M}$ask denoising), a post-training approach that incorporates both temporal and spatial trimming strategies, to accelerate inference without compromising output quality while supporting the inference-time scaling for Gaussian diffusion models. Instead of scaling denoising trajectories in a costly end-to-end manner, we develop a lightweight selector model to evaluate latent Gaussian primitives derived from multiple sampled noises, enabling early trajectory reduction by selecting candidates with high-quality potential. Furthermore, we introduce instance mask denoising to prune learnable Gaussian primitives by filtering out redundant background regions, reducing inference computation at each denoising step. Extensive experiments and analysis demonstrate that TRIM significantly improves both the efficiency and quality of 3D generation. Source code is available at $\href{https://github.com/zeyuanyin/TRIM}{link}$.