C-STEP: Continuous Space-Time Empowerment for Physics-informed Safe Reinforcement Learning of Mobile Agents

TL;DR

C-STEP introduces a physics-informed safety measure for reinforcement learning in mobile agents, enhancing collision avoidance and safety while maintaining task efficiency through intrinsic reward shaping.

Contribution

It proposes a novel continuous space-time empowerment measure tailored for physics-informed safe RL, integrating it into reward functions for improved safety in mobile robotics.

Findings

Fewer collisions observed in experiments

Reduced proximity to obstacles

Marginal increase in travel time

Abstract

Safe navigation in complex environments remains a central challenge for reinforcement learning (RL) in robotics. This paper introduces Continuous Space-Time Empowerment for Physics-informed (C-STEP) safe RL, a novel measure of agent-centric safety tailored to deterministic, continuous domains. This measure can be used to design physics-informed intrinsic rewards by augmenting positive navigation reward functions. The reward incorporates the agents internal states (e.g., initial velocity) and forward dynamics to differentiate safe from risky behavior. By integrating C-STEP with navigation rewards, we obtain an intrinsic reward function that jointly optimizes task completion and collision avoidance. Numerical results demonstrate fewer collisions, reduced proximity to obstacles, and only marginal increases in travel time. Overall, C-STEP offers an interpretable, physics-informed approach to reward shaping in RL, contributing to safety for agentic mobile robotic systems.