Realistic Curriculum Reinforcement Learning for Autonomous and Sustainable Marine Vessel Navigation

TL;DR

This paper introduces a realistic curriculum reinforcement learning framework for autonomous marine vessel navigation, integrating data-driven simulation, emission prediction, and safety considerations to enhance sustainability and safety in maritime transport.

Contribution

It presents a novel CRL framework with a realistic simulation environment and fuel consumption modeling, advancing autonomous navigation with sustainability focus.

Findings

Effective in complex maritime scenarios

Improves safety and emission reduction

Ensures stable learning in continuous actions

Abstract

Sustainability is becoming increasingly critical in the maritime transport, encompassing both environmental and social impacts, such as Greenhouse Gas (GHG) emissions and navigational safety. Traditional vessel navigation heavily relies on human experience, often lacking autonomy and emission awareness, and is prone to human errors that may compromise safety. In this paper, we propose a Curriculum Reinforcement Learning (CRL) framework integrated with a realistic, data-driven marine simulation environment and a machine learning-based fuel consumption prediction module. The simulation environment is constructed using real-world vessel movement data and enhanced with a Diffusion Model to simulate dynamic maritime conditions. Vessel fuel consumption is estimated using historical operational data and learning-based regression. The surrounding environment is represented as image-based inputs to capture spatial complexity. We design a lightweight, policy-based CRL agent with a comprehensive reward mechanism that considers safety, emissions, timeliness, and goal completion. This framework effectively handles complex tasks progressively while ensuring stable and efficient learning in continuous action spaces. We validate the proposed approach in a sea area of the Indian Ocean, demonstrating its efficacy in enabling sustainable and safe vessel navigation.