Multi Actor-Critic DDPG for Robot Action Space Decomposition: A Framework to Control Large 3D Deformation of Soft Linear Objects

TL;DR

This paper introduces MultiAC6, a multi-agent deep reinforcement learning framework that effectively controls large 3D deformations of deformable linear objects, overcoming modeling challenges and achieving high success rates in real-world robotic manipulation.

Contribution

The paper presents a novel multi Actor-Critic DRL framework for DLO manipulation, enabling large deformations and generalization across different DLO types without retraining.

Findings

Achieves up to 40 cm deformation in real-world experiments.

66% higher success rate than single-agent approaches.

Generalizes well to different DLO lengths and materials.

Abstract

Robotic manipulation of deformable linear objects (DLOs) has great potential for applications in diverse fields such as agriculture or industry. However, a major challenge lies in acquiring accurate deformation models that describe the relationship between robot motion and DLO deformations. Such models are difficult to calculate analytically and vary among DLOs. Consequently, manipulating DLOs poses significant challenges, particularly in achieving large deformations that require highly accurate global models. To address these challenges, this paper presents MultiAC6: a new multi Actor-Critic framework for robot action space decomposition to control large 3D deformations of DLOs. In our approach, two deep reinforcement learning (DRL) agents orient and position a robot gripper to deform a DLO into the desired shape. Unlike previous DRL-based studies, MultiAC6 is able to solve the sim-to-real gap, achieving large 3D deformations up to 40 cm in real-world settings. Experimental results also show that MultiAC6 has a 66\% higher success rate than a single-agent approach. Further experimental studies demonstrate that MultiAC6 generalizes well, without retraining, to DLOs with different lengths or materials.