Solving Minimum-Cost Reach Avoid using Reinforcement Learning

TL;DR

This paper introduces RC-PPO, a reinforcement learning method that effectively solves the minimum-cost reach-avoid problem by leveraging Hamilton-Jacobi reachability, outperforming existing methods in goal-reaching and cost minimization.

Contribution

The paper proposes RC-PPO, a novel RL approach that directly addresses the minimum-cost reach-avoid problem, overcoming limitations of surrogate objectives.

Findings

RC-PPO achieves up to 57% lower cumulative costs.

RC-PPO maintains goal-reaching rates comparable to existing methods.

Empirical results are demonstrated on Mujoco benchmarks.

Abstract

Current reinforcement-learning methods are unable to directly learn policies that solve the minimum cost reach-avoid problem to minimize cumulative costs subject to the constraints of reaching the goal and avoiding unsafe states, as the structure of this new optimization problem is incompatible with current methods. Instead, a surrogate problem is solved where all objectives are combined with a weighted sum. However, this surrogate objective results in suboptimal policies that do not directly minimize the cumulative cost. In this work, we propose RC-PPO, a reinforcement-learning-based method for solving the minimum-cost reach-avoid problem by using connections to Hamilton-Jacobi reachability. Empirical results demonstrate that RC-PPO learns policies with comparable goal-reaching rates to while achieving up to 57% lower cumulative costs compared to existing methods on a suite of minimum-cost reach-avoid benchmarks on the Mujoco simulator. The project page can be found at https://oswinso.xyz/rcppo.