Vessel-following model for inland waterways based on deep reinforcement learning

TL;DR

This paper develops a deep reinforcement learning-based vessel-following model for inland waterways, demonstrating its ability to handle complex dynamics and environmental disturbances with high safety and comfort in unseen scenarios.

Contribution

It introduces a novel RL-based vessel-following model that accounts for realistic vessel dynamics and environmental factors, with strong generalization to new scenarios.

Findings

Model ensures safe and comfortable navigation.

Effective damping of traffic oscillations.

High generalization to unseen scenarios.

Abstract

While deep reinforcement learning (RL) has been increasingly applied in designing car-following models in the last years, this study aims at investigating the feasibility of RL-based vehicle-following for complex vehicle dynamics and strong environmental disturbances. As a use case, we developed an inland waterways vessel-following model based on realistic vessel dynamics, which considers environmental influences, such as varying stream velocity and river profile. We extracted natural vessel behavior from anonymized AIS data to formulate a reward function that reflects a realistic driving style next to comfortable and safe navigation. Aiming at high generalization capabilities, we propose an RL training environment that uses stochastic processes to model leading trajectory and river dynamics. To validate the trained model, we defined different scenarios that have not been seen in training, including realistic vessel-following on the Middle Rhine. Our model demonstrated safe and comfortable driving in all scenarios, proving excellent generalization abilities. Furthermore, traffic oscillations could effectively be dampened by deploying the trained model on a sequence of following vessels.