Risk Conditioned Neural Motion Planning

TL;DR

This paper introduces a risk-conditioned deep reinforcement learning approach for motion planning that efficiently produces risk-bounded plans with adjustable risk levels, outperforming traditional methods in complex scenarios.

Contribution

It extends the soft actor critic model with a risk critic to accurately estimate execution risk and allows dynamic risk level adjustment during planning.

Findings

Outperforms mathematical programming baseline in speed and plan quality

Handles nonlinear dynamics and larger state spaces effectively

Provides adjustable risk bounds for flexible planning

Abstract

Risk-bounded motion planning is an important yet difficult problem for safety-critical tasks. While existing mathematical programming methods offer theoretical guarantees in the context of constrained Markov decision processes, they either lack scalability in solving larger problems or produce conservative plans. Recent advances in deep reinforcement learning improve scalability by learning policy networks as function approximators. In this paper, we propose an extension of soft actor critic model to estimate the execution risk of a plan through a risk critic and produce risk-bounded policies efficiently by adding an extra risk term in the loss function of the policy network. We define the execution risk in an accurate form, as opposed to approximating it through a summation of immediate risks at each time step that leads to conservative plans. Our proposed model is conditioned on a continuous spectrum of risk bounds, allowing the user to adjust the risk-averse level of the agent on the fly. Through a set of experiments, we show the advantage of our model in terms of both computational time and plan quality, compared to a state-of-the-art mathematical programming baseline, and validate its performance in more complicated scenarios, including nonlinear dynamics and larger state space.