Contextual Multi-Task Reinforcement Learning for Autonomous Reef Monitoring

TL;DR

This paper introduces a contextual multi-task reinforcement learning approach for autonomous underwater reef monitoring, enabling robots to adapt to various tasks and conditions in uncertain marine environments.

Contribution

It proposes a novel multi-task RL paradigm that improves policy reusability and robustness for diverse reef monitoring tasks in underwater environments.

Findings

The learned policies are sample-efficient and can generalize to unseen tasks without additional training.

The policies demonstrate robustness to water current variations in simulated environments.

Multi-task RL enhances the reusability and effectiveness of autonomous reef monitoring controllers.

Abstract

Although autonomous underwater vehicles promise the capability of marine ecosystem monitoring, their deployment is fundamentally limited by the difficulty of controlling vehicles under highly uncertain and non-stationary underwater dynamics. To address these challenges, we employ a data-driven reinforcement learning approach to compensate for unknown dynamics and task variations.Traditional single-task reinforcement learning has a tendency to overfit the training environment, thus, limit the long-term usefulness of the learnt policy. Hence, we propose to use a contextual multi-task reinforcement learning paradigm instead, allowing us to learn controllers that can be reused for various tasks, e.g., detecting oysters in one reef and detecting corals in another. We evaluate whether contextual multi-task reinforcement learning can efficiently learn robust and generalisable control policies for autonomous underwater reef monitoring. We train a single context-dependent policy that is able to solve multiple related monitoring tasks in a simulated reef environment in HoloOcean. In our experiments, we empirically evaluate the contextual policies regarding sample-efficiency, zero-shot generalisation to unseen tasks, and robustness to varying water currents. By utilising multi-task reinforcement learning, we aim to improve the training effectiveness, as well as the reusability of learnt policies to take a step towards more sustainable procedures in autonomous reef monitoring.