A reinforcement learning path planning approach for range-only underwater target localization with autonomous vehicles

TL;DR

This paper introduces a reinforcement learning approach to optimize the path of autonomous underwater vehicles for target localization, improving accuracy and efficiency over traditional analytical methods.

Contribution

It demonstrates that deep reinforcement learning can learn superior navigation policies for underwater target localization compared to Fisher information matrix-based solutions.

Findings

RL policy reduces median localization error by 17%.

Deep RL outperforms analytical Fisher information matrix approaches.

Potential application in tracking acoustically tagged marine animals.

Abstract

Underwater target localization using range-only and single-beacon (ROSB) techniques with autonomous vehicles has been used recently to improve the limitations of more complex methods, such as long baseline and ultra-short baseline systems. Nonetheless, in ROSB target localization methods, the trajectory of the tracking vehicle near the localized target plays an important role in obtaining the best accuracy of the predicted target position. Here, we investigate a Reinforcement Learning (RL) approach to find the optimal path that an autonomous vehicle should follow in order to increase and optimize the overall accuracy of the predicted target localization, while reducing time and power consumption. To accomplish this objective, different experimental tests have been designed using state-of-the-art deep RL algorithms. Our study also compares the results obtained with the analytical Fisher information matrix approach used in previous studies. The results revealed that the policy learned by the RL agent outperforms trajectories based on these analytical solutions, e.g. the median predicted error at the beginning of the target's localisation is 17% less. These findings suggest that using deep RL for localizing acoustic targets could be successfully applied to in-water applications that include tracking of acoustically tagged marine animals by autonomous underwater vehicles. This is envisioned as a first necessary step to validate the use of RL to tackle such problems, which could be used later on in a more complex scenarios