I-Scene: 3D Instance Models are Implicit Generalizable Spatial Learners

TL;DR

This paper introduces a novel approach that reprograms a pre-trained 3D instance generator to serve as a generalizable scene-level spatial learner, capable of understanding and generating unseen 3D scenes through geometric cues.

Contribution

It presents a method to transform a 3D instance generator into a scene-level model that generalizes to new layouts without dataset-specific supervision.

Findings

The reprogrammed generator can infer spatial relations from geometric cues.

It generalizes to unseen scene layouts and object compositions.

Spatial reasoning emerges even with randomly composed training scenes.

Abstract

Generalization remains the central challenge for interactive 3D scene generation. Existing learning-based approaches ground spatial understanding in limited scene dataset, restricting generalization to new layouts. We instead reprogram a pre-trained 3D instance generator to act as a scene level learner, replacing dataset-bounded supervision with model-centric spatial supervision. This reprogramming unlocks the generator transferable spatial knowledge, enabling generalization to unseen layouts and novel object compositions. Remarkably, spatial reasoning still emerges even when the training scenes are randomly composed objects. This demonstrates that the generator's transferable scene prior provides a rich learning signal for inferring proximity, support, and symmetry from purely geometric cues. Replacing widely used canonical space, we instantiate this insight with a view-centric formulation of the scene space, yielding a fully feed-forward, generalizable scene generator that learns spatial relations directly from the instance model. Quantitative and qualitative results show that a 3D instance generator is an implicit spatial learner and reasoner, pointing toward foundation models for interactive 3D scene understanding and generation. Project page: https://luling06.github.io/I-Scene-project/