Robust Perception-Based Navigation using PAC-NMPC with a Learned Value Function

TL;DR

This paper introduces a robust perception-based navigation method combining PAC-NMPC with a learned value function, enhancing safety and long-term performance in complex environments for both simulation and real-world robotic systems.

Contribution

It integrates a learned value function into PAC-NMPC to improve safety and long-term navigation performance, with proven effectiveness in simulation and real-world tests.

Findings

Enhanced safety in simulation and real-world navigation.

Improved long-term behavior over traditional PAC-NMPC.

Successful real-world deployment on a rally car.

Abstract

Nonlinear model predictive control (NMPC) is typically restricted to short, finite horizons to limit the computational burden of online optimization. As a result, global planning frameworks are frequently necessary to avoid local minima when using NMPC for navigation in complex environments. By contrast, reinforcement learning (RL) can generate policies that minimize the expected cost over an infinite-horizon and can often avoid local minima, even when operating only on current sensor measurements. However, these learned policies are usually unable to provide performance guarantees (e.g., on collision avoidance), especially when outside of the training distribution. In this paper, we augment Probably Approximately Correct NMPC (PAC-NMPC), a sampling-based stochastic NMPC algorithm capable of providing statistical guarantees of performance and safety, with an approximate perception-based value function trained via RL. We demonstrate in simulation that our algorithm can improve the long-term behavior of PAC-NMPC while outperforming other approaches with regards to safety for both planar car dynamics and more complex, high-dimensional fixed-wing aerial vehicle dynamics. We also demonstrate that, even when our value function is trained in simulation, our algorithm can successfully achieve statistically safe navigation on hardware using a 1/10th scale rally car in cluttered real-world environments using only current sensor information.