Rapidly Learning Soft Robot Control via Implicit Time-Stepping

TL;DR

This paper demonstrates that using implicit time-stepping in soft robot simulation significantly accelerates policy learning without losing accuracy, making soft robotics more accessible for reinforcement learning applications.

Contribution

It introduces a general-purpose implicit soft-body simulator, DisMech, and a novel control method called delta natural curvature control, enabling faster and more efficient soft robot policy learning.

Findings

Implicit time-stepping speeds up simulations by up to 40x in contact-rich scenarios.

Parallel simulation of 500 environments is feasible with significant speedups.

Policies trained in one simulator transfer effectively to another, maintaining accuracy.

Abstract

With the explosive growth of rigid-body simulators, policy learning in simulation has become the de facto standard for most rigid morphologies. In contrast, soft robotic simulation frameworks remain scarce and are seldom adopted by the soft robotics community. This gap stems partly from the lack of easy-to-use, general-purpose frameworks and partly from the high computational cost of accurately simulating continuum mechanics, which often renders policy learning infeasible. In this work, we demonstrate that rapid soft robot policy learning is indeed achievable via implicit time-stepping. Our simulator of choice, DisMech, is a general-purpose, fully implicit soft-body simulator capable of handling both soft dynamics and frictional contact. We further introduce delta natural curvature control, a method analogous to delta joint position control in rigid manipulators, providing an intuitive and effective means of enacting control for soft robot learning. To highlight the benefits of implicit time-stepping and delta curvature control, we conduct extensive comparisons across four diverse soft manipulator tasks against one of the most widely used soft-body frameworks, Elastica. With implicit time-stepping, parallel stepping of 500 environments achieves up to 6x faster speeds for non-contact cases and up to 40x faster for contact-rich scenarios. Finally, a comprehensive sim-to-sim gap evaluation--training policies in one simulator and evaluating them in another--demonstrates that implicit time-stepping provides a rare free lunch: dramatic speedups achieved without sacrificing accuracy.