A Scene Graph Backed Approach to Open Set Semantic Mapping

TL;DR

This paper introduces a novel robotic mapping approach that integrates 3D scene graphs as the core knowledge structure, enabling real-time, consistent, and scalable semantic mapping for improved high-level reasoning in large environments.

Contribution

It proposes a scene graph backed mapping architecture that maintains topological consistency and supports hierarchical representations, bridging raw sensor data with symbolic reasoning.

Findings

Real-time scene graph inference and updates during exploration

Enhanced map consistency and scalability in large environments

Facilitation of high-level reasoning with explicit, verifiable structures

Abstract

While Open Set Semantic Mapping and 3D Semantic Scene Graphs (3DSSGs) are established paradigms in robotic perception, deploying them effectively to support high-level reasoning in large-scale, real-world environments remains a significant challenge. Most existing approaches decouple perception from representation, treating the scene graph as a derivative layer generated post hoc. This limits both consistency and scalability. In contrast, we propose a mapping architecture where the 3DSSG serves as the foundational backend, acting as the primary knowledge representation for the entire mapping process. Our approach leverages prior work on incremental scene graph prediction to infer and update the graph structure in real-time as the environment is explored. This ensures that the map remains topologically consistent and computationally efficient, even during extended operations in large-scale settings. By maintaining an explicit, spatially grounded representation that supports both flat and hierarchical topologies, we bridge the gap between sub-symbolic raw sensor data and high-level symbolic reasoning. Consequently, this provides a stable, verifiable structure that knowledge-driven frameworks, ranging from knowledge graphs and ontologies to Large Language Models (LLMs), can directly exploit, enabling agents to operate with enhanced interpretability, trustworthiness, and alignment to human concepts.