Trajectory Planning for Teleoperated Space Manipulators Using Deep Reinforcement Learning

TL;DR

This paper introduces a deep reinforcement learning framework for trajectory planning in teleoperated space manipulators, effectively handling system dynamics complexities and communication delays through innovative methods.

Contribution

The paper presents a novel DRL-based framework with three methods to manage delays and uncertainties in space manipulator trajectory planning, outperforming traditional approaches.

Findings

State Augmentation method shows highest efficiency and robustness.

All three proposed methods effectively handle delays in trajectory planning.

Extensive simulations validate the superiority of the State Augmentation approach.

Abstract

Trajectory planning for teleoperated space manipulators involves challenges such as accurately modeling system dynamics, particularly in free-floating modes with non-holonomic constraints, and managing time delays that increase model uncertainty and affect control precision. Traditional teleoperation methods rely on precise dynamic models requiring complex parameter identification and calibration, while data-driven methods do not require prior knowledge but struggle with time delays. A novel framework utilizing deep reinforcement learning (DRL) is introduced to address these challenges. The framework incorporates three methods: Mapping, Prediction, and State Augmentation, to handle delays when delayed state information is received at the master end. The Soft Actor Critic (SAC) algorithm processes the state information to compute the next action, which is then sent to the remote manipulator for environmental interaction. Four environments are constructed using the MuJoCo simulation platform to account for variations in base and target fixation: fixed base and target, fixed base with rotated target, free-floating base with fixed target, and free-floating base with rotated target. Extensive experiments with both constant and random delays are conducted to evaluate the proposed methods. Results demonstrate that all three methods effectively address trajectory planning challenges, with State Augmentation showing superior efficiency and robustness.