Deep Reinforcement Learning for Mapless Navigation of a Hybrid Aerial Underwater Vehicle with Medium Transition

TL;DR

This paper demonstrates that Deep Reinforcement Learning can effectively enable autonomous mapless navigation and obstacle avoidance for hybrid aerial underwater vehicles, outperforming traditional geometric controllers in complex environments.

Contribution

It introduces two Deep-RL approaches for HUAUV navigation, combining relative localization and sparse range data, and compares their performance with traditional methods.

Findings

Deep-RL approaches successfully enable mapless navigation.

Deep-RL outperforms geometric controllers in obstacle avoidance.

HUAUVs can navigate across air and water media using Deep-RL.

Abstract

Since the application of Deep Q-Learning to the continuous action domain in Atari-like games, Deep Reinforcement Learning (Deep-RL) techniques for motion control have been qualitatively enhanced. Nowadays, modern Deep-RL can be successfully applied to solve a wide range of complex decision-making tasks for many types of vehicles. Based on this context, in this paper, we propose the use of Deep-RL to perform autonomous mapless navigation for Hybrid Unmanned Aerial Underwater Vehicles (HUAUVs), robots that can operate in both, air or water media. We developed two approaches, one deterministic and the other stochastic. Our system uses the relative localization of the vehicle and simple sparse range data to train the network. We compared our approaches with a traditional geometric tracking controller for mapless navigation. Based on experimental results, we can conclude that Deep-RL-based approaches can be successfully used to perform mapless navigation and obstacle avoidance for HUAUVs. Our vehicle accomplished the navigation in two scenarios, being capable to achieve the desired target through both environments, and even outperforming the geometric-based tracking controller on the obstacle-avoidance capability.