Functional 3D Scene Synthesis through Human-Scene Optimization

TL;DR

This paper introduces a human-centric method for generating functional 3D indoor scenes from text, ensuring practical usability by modeling human-object interactions and optimizing scene coherence.

Contribution

It proposes a novel approach that synthesizes 3D scenes conditioned on human interaction, using reasoning, scene assembly, and optimization to improve scene functionality and coherence.

Findings

Generated scenes are more usable and coherent.

The method outperforms existing layout prediction approaches.

Qualitative and quantitative validation confirms effectiveness.

Abstract

This paper presents a novel generative approach that outputs 3D indoor environments solely from a textual description of the scene. Current methods often treat scene synthesis as a mere layout prediction task, leading to rooms with overlapping objects or overly structured scenes, with limited consideration of the practical usability of the generated environment. Instead, our approach is based on a simple, but effective principle: we condition scene synthesis to generate rooms that are usable by humans. This principle is implemented by synthesizing 3D humans that interact with the objects composing the scene. If this human-centric scene generation is viable, the room layout is functional and it leads to a more coherent 3D structure. To this end, we propose a novel method for functional 3D scene synthesis, which consists of reasoning, 3D assembling and optimization. We regard text guided 3D synthesis as a reasoning process by generating a scene graph via a graph diffusion network. Considering object functional co-occurrence, a new strategy is designed to better accommodate human-object interaction and avoidance, achieving human-aware 3D scene optimization. We conduct both qualitative and quantitative experiments to validate the effectiveness of our method in generating coherent 3D scene synthesis results.