CG3D: Compositional Generation for Text-to-3D via Gaussian Splatting

TL;DR

CG3D introduces a novel method for compositional text-to-3D generation using Gaussian radiance fields, enabling detailed, multi-object scenes with physical realism and improved control over object configurations.

Contribution

The paper presents a new explicit Gaussian radiance field approach for scalable, compositional 3D asset generation that overcomes key limitations of previous methods.

Findings

Achieves state-of-the-art results in multi-object scene generation

Outperforms existing models in physics accuracy and object composition

Enables detailed and controllable 3D scene synthesis

Abstract

With the onset of diffusion-based generative models and their ability to generate text-conditioned images, content generation has received a massive invigoration. Recently, these models have been shown to provide useful guidance for the generation of 3D graphics assets. However, existing work in text-conditioned 3D generation faces fundamental constraints: (i) inability to generate detailed, multi-object scenes, (ii) inability to textually control multi-object configurations, and (iii) physically realistic scene composition. In this work, we propose CG3D, a method for compositionally generating scalable 3D assets that resolves these constraints. We find that explicit Gaussian radiance fields, parameterized to allow for compositions of objects, possess the capability to enable semantically and physically consistent scenes. By utilizing a guidance framework built around this explicit representation, we show state of the art results, capable of even exceeding the guiding diffusion model in terms of object combinations and physics accuracy.