Object Manipulation in Marine Environments using Reinforcement Learning

TL;DR

This paper presents a deep reinforcement learning approach using SAC for robust object manipulation in marine environments, effectively handling wave disturbances in simulation.

Contribution

It introduces the application of SAC in maritime object manipulation, demonstrating its effectiveness in dynamic, wave-affected conditions within a realistic simulation.

Findings

Achieved 80% success rate in object retrieval under sea state 2 conditions.

Validated the approach in a realistic maritime simulation environment.

Demonstrated robustness of DRL method against wave disturbances.

Abstract

Performing intervention tasks in the maritime domain is crucial for safety and operational efficiency. The unpredictable and dynamic marine environment makes the intervention tasks such as object manipulation extremely challenging. This study proposes a robust solution for object manipulation from a dock in the presence of disturbances caused by sea waves. To tackle this challenging problem, we apply a deep reinforcement learning (DRL) based algorithm called Soft. Actor-Critic (SAC). SAC employs an actor-critic framework; the actors learn a policy that minimizes an objective function while the critic evaluates the learned policy and provides feedback to guide the actor-learning process. We trained the agent using the PyBullet dynamic simulator and tested it in a realistic simulation environment called MBZIRC maritime simulator. This simulator allows the simulation of different wave conditions according to the World Meteorological Organization (WMO) sea state code. Simulation results demonstrate a high success rate in retrieving the objects from the dock. The trained agent achieved an 80 percent success rate when applied in the simulation environment in the presence of waves characterized by sea state 2, according to the WMO sea state code