Reinforcement Learning based on Scenario-tree MPC for ASVs

TL;DR

This paper introduces a reinforcement learning approach integrated with robust model predictive control to enhance autonomous surface vehicle navigation, obstacle avoidance, and energy efficiency under uncertain environmental conditions.

Contribution

It presents a novel RL-MPC framework that combines scenario-tree robust MPC with Q-learning for adaptive parameter tuning in ASV control.

Findings

Effective obstacle avoidance demonstrated in simulation

Reduced energy consumption and mission time

Robustness to thruster failures and environmental disturbances

Abstract

In this paper, we present the use of Reinforcement Learning (RL) based on Robust Model Predictive Control (RMPC) for the control of an Autonomous Surface Vehicle (ASV). The RL-MPC strategy is utilized for obstacle avoidance and target (set-point) tracking. A scenario-tree robust MPC is used to handle potential failures of the ship thrusters. Besides, the wind and ocean current are considered as unknown stochastic disturbances in the real system, which are handled via constraints tightening. The tightening and other cost parameters are adjusted by RL, using a Q-learning technique. An economic cost is considered, minimizing the time and energy required to achieve the ship missions. The method is illustrated in simulation on a nonlinear 3-DOF model of a scaled version of the Cybership 2.