Self-Curriculum Model-based Reinforcement Learning for Shape Control of Deformable Linear Objects

TL;DR

This paper introduces a novel two-stage reinforcement learning framework with self-curriculum goal generation and visual servoing for precise shape control of deformable linear objects, achieving high success and transferability from simulation to real-world tasks.

Contribution

It presents a new self-curriculum model-based RL approach with ensemble dynamics and visual servoing for efficient, precise, and transferable shape control of DLOs.

Findings

Outperforms baseline methods in success rate and precision

Enables zero-shot transfer from simulation to real-world tasks

Achieves high success across diverse initial and target shapes

Abstract

Precise shape control of Deformable Linear Objects (DLOs) is crucial in robotic applications such as industrial and medical fields. However, existing methods face challenges in handling complex large deformation tasks, especially those involving opposite curvatures, and lack efficiency and precision. To address this, we propose a two-stage framework combining Reinforcement Learning (RL) and online visual servoing. In the large-deformation stage, a model-based reinforcement learning approach using an ensemble of dynamics models is introduced to significantly improve sample efficiency. Additionally, we design a self-curriculum goal generation mechanism that dynamically selects intermediate-difficulty goals with high diversity through imagined evaluations, thereby optimizing the policy learning process. In the small-deformation stage, a Jacobian-based visual servo controller is deployed to ensure high-precision convergence. Simulation results show that the proposed method enables efficient policy learning and significantly outperforms mainstream baselines in shape control success rate and precision. Furthermore, the framework effectively transfers the policy trained in simulation to real-world tasks with zero-shot adaptation. It successfully completes all 30 cases with diverse initial and target shapes across DLOs of different sizes and materials. The project website is available at: https://anonymous.4open.science/w/sc-mbrl-dlo-EB48/