Model-free Neural Lyapunov Control for Safe Robot Navigation

TL;DR

This paper introduces a model-free neural Lyapunov control method that explicitly co-learns safety guarantees with the control policy, enabling safe robot navigation without relying on explicit models or reward shaping.

Contribution

The paper proposes a novel approach that co-learns a Twin Neural Lyapunov Function with the control policy in DRL, providing safety guarantees through a runtime monitor during navigation.

Findings

Outperforms reward-augmented DRL in safety-sensitive tasks

Provides collision-free trajectories with safety guarantees

Effective in high-dimensional navigation scenarios

Abstract

Model-free Deep Reinforcement Learning (DRL) controllers have demonstrated promising results on various challenging non-linear control tasks. While a model-free DRL algorithm can solve unknown dynamics and high-dimensional problems, it lacks safety assurance. Although safety constraints can be encoded as part of a reward function, there still exists a large gap between an RL controller trained with this modified reward and a safe controller. In contrast, instead of implicitly encoding safety constraints with rewards, we explicitly co-learn a Twin Neural Lyapunov Function (TNLF) with the control policy in the DRL training loop and use the learned TNLF to build a runtime monitor. Combined with the path generated from a planner, the monitor chooses appropriate waypoints that guide the learned controller to provide collision-free control trajectories. Our approach inherits the scalability advantages from DRL while enhancing safety guarantees. Our experimental evaluation demonstrates the effectiveness of our approach compared to DRL with augmented rewards and constrained DRL methods over a range of high-dimensional safety-sensitive navigation tasks.