Forest2Seq: Revitalizing Order Prior for Sequential Indoor Scene Synthesis

TL;DR

Forest2Seq introduces an order-aware sequential learning framework that structures indoor scene objects into hierarchical trees, enabling more realistic 3D scene synthesis with improved metrics over existing autoregressive models.

Contribution

It formulates indoor scene synthesis as an order-aware sequential problem and employs a hierarchical scene tree structure with transformer-based generation, a novel approach in this domain.

Findings

Outperforms baselines in FID and KL scores

Effective in downstream tasks and ablation studies

Highlights importance of order prior in scene generation

Abstract

Synthesizing realistic 3D indoor scenes is a challenging task that traditionally relies on manual arrangement and annotation by expert designers. Recent advances in autoregressive models have automated this process, but they often lack semantic understanding of the relationships and hierarchies present in real-world scenes, yielding limited performance. In this paper, we propose Forest2Seq, a framework that formulates indoor scene synthesis as an order-aware sequential learning problem. Forest2Seq organizes the inherently unordered collection of scene objects into structured, ordered hierarchical scene trees and forests. By employing a clustering-based algorithm and a breadth-first traversal, Forest2Seq derives meaningful orderings and utilizes a transformer to generate realistic 3D scenes autoregressively. Experimental results on standard benchmarks demonstrate Forest2Seq's superiority in synthesizing more realistic scenes compared to top-performing baselines, with significant improvements in FID and KL scores. Our additional experiments for downstream tasks and ablation studies also confirm the importance of incorporating order as a prior in 3D scene generation.