A Deep Reinforcement Learning Framework and Methodology for Reducing the Sim-to-Real Gap in ASV Navigation

TL;DR

This paper presents a deep reinforcement learning framework for autonomous surface vehicle navigation that integrates hydrodynamics models and domain randomization to effectively reduce the sim-to-real gap, improving real-world performance.

Contribution

The authors introduce a novel DRL framework incorporating buoyancy and hydrodynamics models, along with system identification and domain randomization, to enhance real-world deployment of ASVs.

Findings

Lowered energy consumption by 13.1% in real-world tests

Reduced task completion time by 7.4%

Demonstrated improved sim-to-real transfer for ASV navigation

Abstract

Despite the increasing adoption of Deep Reinforcement Learning (DRL) for Autonomous Surface Vehicles (ASVs), there still remain challenges limiting real-world deployment. In this paper, we first integrate buoyancy and hydrodynamics models into a modern Reinforcement Learning framework to reduce training time. Next, we show how system identification coupled with domain randomization improves the RL agent performance and narrows the sim-to-real gap. Real-world experiments for the task of capturing floating waste show that our approach lowers energy consumption by 13.1\% while reducing task completion time by 7.4\%. These findings, supported by sharing our open-source implementation, hold the potential to impact the efficiency and versatility of ASVs, contributing to environmental conservation efforts.