HiGS: Hierarchical Generative Scene Framework for Multi-Step Associative Semantic Spatial Composition

TL;DR

HiGS is a hierarchical framework that enables multi-step, user-guided 3D scene generation by combining semantic control with automatic completion, inspired by human scene modeling processes.

Contribution

It introduces a novel hierarchical generative approach with the PHiSSG for structured, coherent, multi-step scene creation with fine-grained user control.

Findings

Outperforms single-stage methods in layout plausibility.

Achieves higher style consistency and user preference.

Supports dynamic, recursive scene layout optimization.

Abstract

Three-dimensional scene generation holds significant potential in gaming, film, and virtual reality. However, most existing methods adopt a single-step generation process, making it difficult to balance scene complexity with minimal user input. Inspired by the human cognitive process in scene modeling, which progresses from global to local, focuses on key elements, and completes the scene through semantic association, we propose HiGS, a hierarchical generative framework for multi-step associative semantic spatial composition. HiGS enables users to iteratively expand scenes by selecting key semantic objects, offering fine-grained control over regions of interest while the model completes peripheral areas automatically. To support structured and coherent generation, we introduce the Progressive Hierarchical Spatial-Semantic Graph (PHiSSG), which dynamically organizes spatial relationships and semantic dependencies across the evolving scene structure. PHiSSG ensures spatial and geometric consistency throughout the generation process by maintaining a one-to-one mapping between graph nodes and generated objects and supporting recursive layout optimization. Experiments demonstrate that HiGS outperforms single-stage methods in layout plausibility, style consistency, and user preference, offering a controllable and extensible paradigm for efficient 3D scene construction.