Censored Deep Reinforcement Patrolling with Information Criterion for Monitoring Large Water Resources using Autonomous Surface Vehicles

TL;DR

This paper introduces a deep reinforcement learning framework with a novel Q-Censoring mechanism for autonomous surface vehicles to efficiently monitor large water resources, reducing redundancy and improving contamination detection accuracy.

Contribution

It presents a new deep Q-learning algorithm with Q-Censoring for obstacle avoidance and information gathering, outperforming previous coverage strategies in water resource monitoring.

Findings

Reduced path redundancy by 3 times using noise-networks.

Achieved 13% higher accuracy in contamination modeling.

Improved dangerous contamination peak detection by 37%.

Abstract

Monitoring and patrolling large water resources is a major challenge for conservation. The problem of acquiring data of an underlying environment that usually changes within time involves a proper formulation of the information. The use of Autonomous Surface Vehicles equipped with water quality sensor modules can serve as an early-warning system agents for contamination peak-detection, algae blooms monitoring, or oil-spill scenarios. In addition to information gathering, the vehicle must plan routes that are free of obstacles on non-convex maps. This work proposes a framework to obtain a collision-free policy that addresses the patrolling task for static and dynamic scenarios. Using information gain as a measure of the uncertainty reduction over data, it is proposed a Deep Q-Learning algorithm improved by a Q-Censoring mechanism for model-based obstacle avoidance. The obtained results demonstrate the usefulness of the proposed algorithm for water resource monitoring for static and dynamic scenarios. Simulations showed the use of noise-networks are a good choice for enhanced exploration, with 3 times less redundancy in the paths. Previous coverage strategies are also outperformed both in the accuracy of the obtained contamination model by a 13% on average and by a 37% in the detection of dangerous contamination peaks. Finally, these results indicate the appropriateness of the proposed framework for monitoring scenarios with autonomous vehicles.