Towards High-Fidelity Single-view Holistic Reconstruction of Indoor Scenes

TL;DR

This paper introduces a novel framework for high-fidelity 3D reconstruction of indoor scenes from a single image, accurately modeling both objects and background with implicit functions and attention mechanisms.

Contribution

It proposes an instance-aligned implicit function and attention module for detailed object reconstruction and a new implicit method for background geometry, improving overall scene reconstruction quality.

Findings

Outperforms existing methods on multiple datasets

Achieves detailed and accurate 3D reconstructions

Effectively handles occlusions in indoor scenes

Abstract

We present a new framework to reconstruct holistic 3D indoor scenes including both room background and indoor objects from single-view images. Existing methods can only produce 3D shapes of indoor objects with limited geometry quality because of the heavy occlusion of indoor scenes. To solve this, we propose an instance-aligned implicit function (InstPIFu) for detailed object reconstruction. Combining with instance-aligned attention module, our method is empowered to decouple mixed local features toward the occluded instances. Additionally, unlike previous methods that simply represents the room background as a 3D bounding box, depth map or a set of planes, we recover the fine geometry of the background via implicit representation. Extensive experiments on the SUN RGB-D, Pix3D, 3D-FUTURE, and 3D-FRONT datasets demonstrate that our method outperforms existing approaches in both background and foreground object reconstruction. Our code and model will be made publicly available.