A reinforcement learning control approach for underwater manipulation under position and torque constraints

TL;DR

This paper introduces a reinforcement learning-based control method for underwater manipulators that effectively manages position and torque constraints, demonstrating adaptability and improved performance in simulation.

Contribution

A novel actor-critic reinforcement learning controller for underwater manipulators that handles constraints and adapts to environmental uncertainties.

Findings

Effective control under constraints demonstrated in simulation

Enhanced adaptability to environmental disturbances

Improved position accuracy in underwater manipulation

Abstract

In marine operations underwater manipulators play a primordial role. However, due to uncertainties in the dynamic model and disturbances caused by the environment, low-level control methods require great capabilities to adapt to change. Furthermore, under position and torque constraints the requirements for the control system are greatly increased. Reinforcement learning is a data driven control technique that can learn complex control policies without the need of a model. The learning capabilities of these type of agents allow for great adaptability to changes in the operative conditions. In this article we present a novel reinforcement learning low-level controller for the position control of an underwater manipulator under torque and position constraints. The reinforcement learning agent is based on an actor-critic architecture using sensor readings as state information. Simulation results using the Reach Alpha 5 underwater manipulator show the advantages of the proposed control strategy.