Generative 3D Gaussians with Learned Density Control

TL;DR

This paper introduces DeG, a flexible 3D representation using learnable density-controlled Gaussians, enabling high-quality generative 3D synthesis with adaptive detail and scalable decoding.

Contribution

DeG models Gaussian centers via a learnable density over an octree, allowing adaptive density control and variable-resolution decoding from a single latent code.

Findings

Achieves state-of-the-art quality in single-image-to-3D generation.

Supports variable-resolution decoding by adjusting sampling budget.

Introduces VecSeq for robust sequence modeling of unordered set latents.

Abstract

We present Density-Sampled Gaussians (DeG), a novel 3D representation designed to bridge the gap between adaptive rendering primitives and scalable generative modeling. Unlike existing approaches that constrain 3D Gaussians to fixed voxel grids or arrays, DeG models Gaussian centers as samples from a learnable probability density function defined over an octree. This formulation provides a rigorous mathematical framework for adaptive density control: by jointly optimizing the spatial density and Gaussian attributes under rendering supervision, our model naturally concentrates primitives in regions of high geometric complexity. We achieve this via a new render loss contribution gradient that serves as a fully differentiable analogue to the discrete densification and pruning heuristics used in standard Gaussian Splatting. The resulting representation is highly flexible, supporting variable-resolution decoding from a single latent code by simply adjusting the sampling budget. To enable generative synthesis, we train a latent diffusion model on DeG. We identify a critical challenge in applying diffusion to unordered set-structured latents, which can significantly slow convergence, and propose VecSeq, a canonical re-indexing mechanism that anchors latent tokens to a deterministic 3D Sobol sequence. This transforms the ambiguous set-generation problem into a robust sequence modeling task. Extensive experiments demonstrate that our pipeline achieves state-of-the-art quality in single-image-to-3D generation, combining the structural adaptivity of unstructured primitives with the training stability of grid-based methods.