Ocean Diviner: A Diffusion-Augmented Reinforcement Learning Framework for AUV Robust Control in Underwater Tasks

TL;DR

This paper introduces a diffusion-augmented reinforcement learning framework that significantly enhances the robustness and adaptability of autonomous underwater vehicles in complex, uncertain marine environments through innovative action generation and hybrid learning strategies.

Contribution

It presents a novel diffusion-based action generation method combined with a hybrid learning architecture, improving long-horizon planning and exploration efficiency for AUV control.

Findings

Outperforms traditional control methods in simulation under challenging conditions

Demonstrates higher robustness and flexibility in dynamic underwater environments

Achieves better exploration efficiency and policy stability

Abstract

Autonomous Underwater Vehicles (AUVs) are essential for marine exploration, yet their control remains highly challenging due to nonlinear dynamics and uncertain environmental disturbances. This paper presents a diffusion-augmented Reinforcement Learning (RL) framework for robust AUV control, aiming to improve AUV's adaptability in dynamic underwater environments. The proposed framework integrates two core innovations: (1) A diffusion-based action generation framework that produces physically feasible and high-quality actions, enhanced by a high-dimensional state encoding mechanism combining current observations with historical states and actions through a novel diffusion U-Net architecture, significantly improving long-horizon planning capacity for robust control. (2) A sample-efficient hybrid learning architecture that synergizes diffusion-guided exploration with RL policy optimization, where the diffusion model generates diverse candidate actions and the RL critic selects the optimal action, achieving higher exploration efficiency and policy stability in dynamic underwater environments. Extensive simulation experiments validate the framework's superior robustness and flexibility, outperforming conventional control methods in challenging marine conditions, offering enhanced adaptability and reliability for AUV operations in underwater tasks. Finally, we will release the code publicly soon to support future research in this area.