OmniIndoor3D: Comprehensive Indoor 3D Reconstruction

TL;DR

OmniIndoor3D introduces a comprehensive indoor 3D reconstruction framework using Gaussian representations, enabling accurate appearance, geometry, and panoptic scene understanding from RGB-D data, with noise reduction and densification strategies.

Contribution

The paper presents a novel Gaussian-based framework that integrates multi-view RGB-D data, a lightweight MLP for geometry refinement, and panoptic-guided densification for improved indoor scene reconstruction.

Findings

Achieves state-of-the-art results in appearance, geometry, and panoptic reconstruction.

Effectively reduces noise in indoor scene geometry.

Provides robust indoor scene understanding for robotic navigation.

Abstract

We propose a novel framework for comprehensive indoor 3D reconstruction using Gaussian representations, called OmniIndoor3D. This framework enables accurate appearance, geometry, and panoptic reconstruction of diverse indoor scenes captured by a consumer-level RGB-D camera. Since 3DGS is primarily optimized for photorealistic rendering, it lacks the precise geometry critical for high-quality panoptic reconstruction. Therefore, OmniIndoor3D first combines multiple RGB-D images to create a coarse 3D reconstruction, which is then used to initialize the 3D Gaussians and guide the 3DGS training. To decouple the optimization conflict between appearance and geometry, we introduce a lightweight MLP that adjusts the geometric properties of 3D Gaussians. The introduced lightweight MLP serves as a low-pass filter for geometry reconstruction and significantly reduces noise in indoor scenes. To improve the distribution of Gaussian primitives, we propose a densification strategy guided by panoptic priors to encourage smoothness on planar surfaces. Through the joint optimization of appearance, geometry, and panoptic reconstruction, OmniIndoor3D provides comprehensive 3D indoor scene understanding, which facilitates accurate and robust robotic navigation. We perform thorough evaluations across multiple datasets, and OmniIndoor3D achieves state-of-the-art results in appearance, geometry, and panoptic reconstruction. We believe our work bridges a critical gap in indoor 3D reconstruction. The code will be released at: https://ucwxb.github.io/OmniIndoor3D/