Closing the Perception-Action Loop for Semantically Safe Navigation in Semi-Static Environments

TL;DR

This paper introduces a perception-action pipeline that uses semantic mapping and control barrier functions to enable autonomous robots to navigate safely in semi-static, changing environments by adapting to scene changes in real-time.

Contribution

It presents a novel closed-loop system combining dense semantic mapping, CBFs, and MPC for safe, adaptive robot navigation in dynamic environments.

Findings

Effective handling of scene changes improves safety.

Semantic information enhances navigation accuracy.

System validated in both simulations and real-world tests.

Abstract

Autonomous robots navigating in changing environments demand adaptive navigation strategies for safe long-term operation. While many modern control paradigms offer theoretical guarantees, they often assume known extrinsic safety constraints, overlooking challenges when deployed in real-world environments where objects can appear, disappear, and shift over time. In this paper, we present a closed-loop perception-action pipeline that bridges this gap. Our system encodes an online-constructed dense map, along with object-level semantic and consistency estimates into a control barrier function (CBF) to regulate safe regions in the scene. A model predictive controller (MPC) leverages the CBF-based safety constraints to adapt its navigation behaviour, which is particularly crucial when potential scene changes occur. We test the system in simulations and real-world experiments to demonstrate the impact of semantic information and scene change handling on robot behavior, validating the practicality of our approach.