UW-MARL: Multi-Agent Reinforcement Learning for Underwater Adaptive Sampling using Autonomous Vehicles

TL;DR

This paper introduces a multi-agent reinforcement learning approach for autonomous underwater vehicles to perform adaptive water-quality sampling efficiently in uncertain environments, reducing energy and time costs.

Contribution

It presents a novel MARL-based adaptive sampling algorithm for multiple autonomous vehicles, optimized for real-world water-quality monitoring tasks.

Findings

Effective in real river environments

Reduces sampling time and energy consumption

Demonstrates improved decision-making in adaptive sampling

Abstract

Near-real-time water-quality monitoring in uncertain environments such as rivers, lakes, and water reservoirs of different variables is critical to protect the aquatic life and to prevent further propagation of the potential pollution in the water. In order to measure the physical values in a region of interest, adaptive sampling is helpful as an energy- and time-efficient technique since an exhaustive search of an area is not feasible with a single vehicle. We propose an adaptive sampling algorithm using multiple autonomous vehicles, which are well-trained, as agents, in a Multi-Agent Reinforcement Learning (MARL) framework to make efficient sequence of decisions on the adaptive sampling procedure. The proposed solution is evaluated using experimental data, which is fed into a simulation framework. Experiments were conducted in the Raritan River, Somerset and in Carnegie Lake, Princeton, NJ during July 2019.