Learning Partially Structured Environmental Dynamics for Marine Robotic Navigation

TL;DR

This paper presents a deep reinforcement learning approach enabling marine robots to navigate effectively in partially structured, dynamic ocean environments by learning to handle complex disturbances like waves and currents.

Contribution

The paper introduces a novel deep reinforcement learning method tailored for marine navigation in partially structured, dynamic environments, addressing the challenge of modeling ocean disturbances.

Findings

Robots trained with the method succeed in complex ocean scenarios.

The approach effectively handles both artificial and real ocean disturbances.

Navigation performance improves in spatiotemporal environments.

Abstract

We investigate the scenario that a robot needs to reach a designated goal after taking a sequence of appropriate actions in a non-static environment that is partially structured. One application example is to control a marine vehicle to move in the ocean. The ocean environment is dynamic and oftentimes the ocean waves result in strong disturbances that can disturb the vehicle's motion. Modeling such dynamic environment is non-trivial, and integrating such model in the robotic motion control is particularly difficult. Fortunately, the ocean currents usually form some local patterns (e.g. vortex) and thus the environment is partially structured. The historically observed data can be used to train the robot to learn to interact with the ocean tidal disturbances. In this paper we propose a method that applies the deep reinforcement learning framework to learn such partially structured complex disturbances. Our results show that, by training the robot under artificial and real ocean disturbances, the robot is able to successfully act in complex and spatiotemporal environments.