Reinforcement Learning-Based Automatic Berthing System

TL;DR

This paper introduces a reinforcement learning-based automatic ship berthing system using PPO, which learns optimal control policies through interaction, eliminating the need for extensive pre-collected training data and demonstrating promising real-world applicability.

Contribution

The study proposes a novel PPO-based berthing system that overcomes data collection limitations of neural network approaches by learning through trial-and-error.

Findings

The PPO-based system effectively controls ship RPS and rudder angle.

It eliminates the need for pre-existing berthing data.

Shows potential for real-world berthing applications.

Abstract

Previous studies on automatic berthing systems based on artificial neural network (ANN) showed great berthing performance by training the ANN with ship berthing data as training data. However, because the ANN requires a large amount of training data to yield robust performance, the ANN-based automatic berthing system is somewhat limited due to the difficulty in obtaining the berthing data. In this study, to overcome this difficulty, the automatic berthing system based on one of the reinforcement learning (RL) algorithms, proximal policy optimization (PPO), is proposed because the RL algorithms can learn an optimal control policy through trial-and-error by interacting with a given environment and does not require any pre-obtained training data, where the control policy in the proposed PPO-based automatic berthing system controls revolutions per second (RPS) and rudder angle of a ship. Finally, it is shown that the proposed PPO-based automatic berthing system eliminates the need for obtaining the training dataset and shows great potential for the actual berthing application.