FurnSet: Exploiting Repeats for 3D Scene Reconstruction

TL;DR

FurnSet is a novel framework that enhances single-view 3D scene reconstruction by explicitly exploiting repeated object instances through set-aware attention and joint optimization.

Contribution

It introduces a set-aware self-attention mechanism and combined scene-object conditioning to leverage object repetitions for improved reconstruction accuracy.

Findings

Improved reconstruction quality on 3D-Future and 3D-Front datasets.

Effective grouping and aggregation of identical object instances.

Enhanced scene alignment through combined 3D and 2D losses.

Abstract

Single-view 3D scene reconstruction involves inferring both object geometry and spatial layout. Existing methods typically reconstruct objects independently or rely on implicit scene context, failing to exploit the repeated instances commonly present in realworld scenes. We propose FurnSet, a framework that explicitly identifies and leverages repeated object instances to improve reconstruction. Our method introduces per-object CLS tokens and a set-aware self-attention mechanism that groups identical instances and aggregates complementary observations across them, enabling joint reconstruction. We further combine scene-level and object-level conditioning to guide object reconstruction, followed by layout optimization using object point clouds with 3D and 2D projection losses for scene alignment. Experiments on 3D-Future and 3D-Front demonstrate improved scene reconstruction quality, highlighting the effectiveness of exploiting repetition for robust 3D scene reconstruction.