Online Approximate Optimal Station Keeping of an Autonomous Underwater Vehicle

TL;DR

This paper presents an online reinforcement learning approach for optimal station keeping of an autonomous underwater vehicle, ensuring bounded convergence without persistent excitation.

Contribution

It introduces a novel actor-critic framework that approximates the solution to a zero-sum game for vehicle control, guaranteeing convergence.

Findings

Guarantees UUB convergence of states and policies

Does not require persistence of excitation

Effective in real-time control scenarios

Abstract

Online approximation of an optimal station keeping strategy for a fully actuated six degrees-of-freedom autonomous underwater vehicle is considered. The developed controller is an approximation of the solution to a two player zero-sum game where the controller is the minimizing player and an external disturbance is the maximizing player. The solution is approximated using a reinforcement learning-based actor-critic framework. The result guarantees uniformly ultimately bounded (UUB) convergence of the states and UUB convergence of the approximated policies to the optimal polices without the requirement of persistence of excitation.