BloomScene: Lightweight Structured 3D Gaussian Splatting for Crossmodal Scene Generation

TL;DR

BloomScene introduces a lightweight, structured 3D Gaussian splatting method for crossmodal scene generation, achieving high-quality, coherent 3D scenes from text or images with reduced storage and enhanced realism.

Contribution

The paper presents a novel framework combining hierarchical depth regularization and structured hash grid compression for efficient, high-quality 3D scene generation from crossmodal inputs.

Findings

Outperforms baseline methods in scene quality and coherence

Reduces storage overhead significantly

Enhances scene realism through depth regularization

Abstract

With the widespread use of virtual reality applications, 3D scene generation has become a new challenging research frontier. 3D scenes have highly complex structures and need to ensure that the output is dense, coherent, and contains all necessary structures. Many current 3D scene generation methods rely on pre-trained text-to-image diffusion models and monocular depth estimators. However, the generated scenes occupy large amounts of storage space and often lack effective regularisation methods, leading to geometric distortions. To this end, we propose BloomScene, a lightweight structured 3D Gaussian splatting for crossmodal scene generation, which creates diverse and high-quality 3D scenes from text or image inputs. Specifically, a crossmodal progressive scene generation framework is proposed to generate coherent scenes utilizing incremental point cloud reconstruction and 3D Gaussian splatting. Additionally, we propose a hierarchical depth prior-based regularization mechanism that utilizes multi-level constraints on depth accuracy and smoothness to enhance the realism and continuity of the generated scenes. Ultimately, we propose a structured context-guided compression mechanism that exploits structured hash grids to model the context of unorganized anchor attributes, which significantly eliminates structural redundancy and reduces storage overhead. Comprehensive experiments across multiple scenes demonstrate the significant potential and advantages of our framework compared with several baselines.