GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splatting

TL;DR

GALA3D introduces a novel framework for text-to-3D complex scene generation using layout-guided Gaussian splatting, combining large language models and diffusion techniques for realistic, controllable 3D scene synthesis.

Contribution

It presents a new layout-guided 3D Gaussian representation and an optimization mechanism for high-fidelity, controllable scene-level 3D content generation from text.

Findings

State-of-the-art scene-level 3D generation quality

Effective control over object placement and interactions

High fidelity and realistic 3D scene synthesis

Abstract

We present GALA3D, generative 3D GAussians with LAyout-guided control, for effective compositional text-to-3D generation. We first utilize large language models (LLMs) to generate the initial layout and introduce a layout-guided 3D Gaussian representation for 3D content generation with adaptive geometric constraints. We then propose an instance-scene compositional optimization mechanism with conditioned diffusion to collaboratively generate realistic 3D scenes with consistent geometry, texture, scale, and accurate interactions among multiple objects while simultaneously adjusting the coarse layout priors extracted from the LLMs to align with the generated scene. Experiments show that GALA3D is a user-friendly, end-to-end framework for state-of-the-art scene-level 3D content generation and controllable editing while ensuring the high fidelity of object-level entities within the scene. The source codes and models will be available at gala3d.github.io.