IRS: Instance-Level 3D Scene Graphs via Room Prior Guided LiDAR-Camera Fusion

TL;DR

This paper introduces a fast, robust method for constructing detailed 3D scene graphs of indoor environments by fusing LiDAR and camera data, leveraging room priors and visual foundation models for improved semantic understanding.

Contribution

It presents a novel framework combining LiDAR-camera fusion with room priors and multi-level VFMs to enhance 3D scene graph construction speed and accuracy in indoor scenes.

Findings

Achieves up to ten times faster scene graph construction than previous methods.

Maintains high semantic precision with improved robustness.

Validates effectiveness through experiments in simulated and real environments.

Abstract

Indoor scene understanding remains a fundamental challenge in robotics, with direct implications for downstream tasks such as navigation and manipulation. Traditional approaches often rely on closed-set recognition or loop closure, limiting their adaptability in open-world environments. With the advent of visual foundation models (VFMs), open-vocabulary recognition and natural language querying have become feasible, unlocking new possibilities for 3D scene graph construction. In this paper, we propose a robust and efficient framework for instance-level 3D scene graph construction via LiDAR-camera fusion. Leveraging LiDAR's wide field of view (FOV) and long-range sensing capabilities, we rapidly acquire room-level geometric priors. Multi-level VFMs are employed to improve the accuracy and consistency of semantic extraction. During instance fusion, room-based segmentation enables parallel processing, while the integration of geometric and semantic cues significantly enhances fusion accuracy and robustness. Compared to state-of-the-art methods, our approach achieves up to an order-of-magnitude improvement in construction speed while maintaining high semantic precision. Extensive experiments in both simulated and real-world environments validate the effectiveness of our approach. We further demonstrate its practical value through a language-guided semantic navigation task, highlighting its potential for real-world robotic applications.