SPARK: Graph-Based Online Semantic Integration System for Robot Task Planning

TL;DR

SPARK is a system that enables robots to update and utilize semantic scene graphs in real-time, improving task planning in dynamic environments by integrating semantic cues online.

Contribution

This work introduces SPARK, a novel framework for online semantic scene graph updates, extending previous offline methods to real-time robot task planning.

Findings

Semantic scene graphs improve robot adaptability in dynamic settings.

Online updates of semantic information enhance task planning accuracy.

Graph-based spatial relationships aid in handling unconventional cues like gestures.

Abstract

The ability to update information acquired through various means online during task execution is crucial for a general-purpose service robot. This information includes geometric and semantic data. While SLAM handles geometric updates on 2D maps or 3D point clouds, online updates of semantic information remain unexplored. We attribute the challenge to the online scene graph representation, for its utility and scalability. Building on previous works regarding offline scene graph representations, we study online graph representations of semantic information in this work. We introduce SPARK: Spatial Perception and Robot Knowledge Integration. This framework extracts semantic information from environment-embedded cues and updates the scene graph accordingly, which is then used for subsequent task planning. We demonstrate that graph representations of spatial relationships enhance the robot system's ability to perform tasks in dynamic environments and adapt to unconventional spatial cues, like gestures.