An Efficient Learning Control Framework With Sim-to-Real for String-Type Artificial Muscle-Driven Robotic Systems

TL;DR

This paper introduces an efficient reinforcement learning framework with sim-to-real transfer for controlling string-type artificial muscle-driven robots, significantly improving data efficiency and bridging the simulation-to-reality gap.

Contribution

It proposes novel bootstrap and augmentation methods for data-efficient DRL and a muscle dynamics randomization technique for effective sim-to-real transfer in artificial muscle robots.

Findings

Enhanced control performance in artificial muscle robots

Reduced sim-to-real transfer gap through muscle dynamics randomization

Validated effectiveness on robotic eye and wrist systems

Abstract

Robotic systems driven by artificial muscles present unique challenges due to the nonlinear dynamics of actuators and the complex designs of mechanical structures. Traditional model-based controllers often struggle to achieve desired control performance in such systems. Deep reinforcement learning (DRL), a trending machine learning technique widely adopted in robot control, offers a promising alternative. However, integrating DRL into these robotic systems faces significant challenges, including the requirement for large amounts of training data and the inevitable sim-to-real gap when deployed to real-world robots. This paper proposes an efficient reinforcement learning control framework with sim-to-real transfer to address these challenges. Bootstrap and augmentation enhancements are designed to improve the data efficiency of baseline DRL algorithms, while a sim-to-real transfer technique, namely randomization of muscle dynamics, is adopted to bridge the gap between simulation and real-world deployment. Extensive experiments and ablation studies are conducted utilizing two string-type artificial muscle-driven robotic systems including a two degree-of-freedom robotic eye and a parallel robotic wrist, the results of which demonstrate the effectiveness of the proposed learning control strategy.