Trajectory Tracking of Underactuated Sea Vessels With Uncertain Dynamics: An Integral Reinforcement Learning Approach

TL;DR

This paper introduces an integral reinforcement learning method for trajectory tracking of underactuated sea vessels with uncertain dynamics, enabling adaptive control with limited prior knowledge.

Contribution

It proposes a novel online reinforcement learning approach using adaptive critics and gradient descent for nonlinear vessel trajectory tracking with uncertain dynamics.

Findings

Effective online learning in various tracking scenarios

Improved control performance under uncertain conditions

Adaptive critic-based solution for nonlinear systems

Abstract

Underactuated systems like sea vessels have degrees of motion that are insufficiently matched by a set of independent actuation forces. In addition, the underlying trajectory-tracking control problems grow in complexity in order to decide the optimal rudder and thrust control signals. This enforces several difficult-to-solve constraints that are associated with the error dynamical equations using classical optimal tracking and adaptive control approaches. An online machine learning mechanism based on integral reinforcement learning is proposed to find a solution for a class of nonlinear tracking problems with partial prior knowledge of the system dynamics. The actuation forces are decided using innovative forms of temporal difference equations relevant to the vessel's surge and angular velocities. The solution is implemented using an online value iteration process which is realized by employing means of the adaptive critics and gradient descent approaches. The adaptive learning mechanism exhibited well-functioning and interactive features in react to different desired reference-tracking scenarios.