Robust Unmanned Surface Vehicle Navigation with Distributional Reinforcement Learning

TL;DR

This paper introduces a distributional reinforcement learning approach for USV navigation that effectively manages environmental uncertainties, leading to safer and more efficient path planning in marine environments with currents.

Contribution

It presents a novel distributional RL-based local planner with adaptive risk sensitivity tuning, outperforming traditional methods in robustness and efficiency.

Findings

Outperforms traditional RL and classical methods in safety and efficiency

Achieves stable learning and converges to safer policies

Robust against environmental flows in marine navigation

Abstract

Autonomous navigation of Unmanned Surface Vehicles (USV) in marine environments with current flows is challenging, and few prior works have addressed the sensorbased navigation problem in such environments under no prior knowledge of the current flow and obstacles. We propose a Distributional Reinforcement Learning (RL) based local path planner that learns return distributions which capture the uncertainty of action outcomes, and an adaptive algorithm that automatically tunes the level of sensitivity to the risk in the environment. The proposed planner achieves a more stable learning performance and converges to safer policies than a traditional RL based planner. Computational experiments demonstrate that comparing to a traditional RL based planner and classical local planning methods such as Artificial Potential Fields and the Bug Algorithm, the proposed planner is robust against environmental flows, and is able to plan trajectories that are superior in safety, time and energy consumption.