DoCRL: Double Critic Deep Reinforcement Learning for Mapless Navigation of a Hybrid Aerial Underwater Vehicle with Medium Transition

TL;DR

This paper introduces a Double Critic Deep Reinforcement Learning method for hybrid aerial-underwater robots, improving their mapless navigation and medium transition capabilities using recurrent neural networks and range data.

Contribution

It proposes a novel Double Critic Actor-Critic approach tailored for hybrid aerial-underwater vehicles, enhancing navigation and transition performance over previous methods.

Findings

Outperforms previous navigation approaches

Improves transition capabilities between air and water

Utilizes recurrent neural networks with range data

Abstract

Deep Reinforcement Learning (Deep-RL) techniques for motion control have been continuously used to deal with decision-making problems for a wide variety of robots. Previous works showed that Deep-RL can be applied to perform mapless navigation, including the medium transition of Hybrid Unmanned Aerial Underwater Vehicles (HUAUVs). These are robots that can operate in both air and water media, with future potential for rescue tasks in robotics. This paper presents new approaches based on the state-of-the-art Double Critic Actor-Critic algorithms to address the navigation and medium transition problems for a HUAUV. We show that double-critic Deep-RL with Recurrent Neural Networks using range data and relative localization solely improves the navigation performance of HUAUVs. Our DoCRL approaches achieved better navigation and transitioning capability, outperforming previous approaches.