Surrogate compliance modeling enables reinforcement learned locomotion gaits for soft robots

TL;DR

This paper introduces a surrogate compliance modeling method that enables reinforcement learning of soft robot locomotion gaits in rigid-body simulators, achieving high-fidelity sim-to-real transfer and robust multi-terrain performance.

Contribution

The authors develop a surrogate compliance modeling approach that captures soft-material deformation effects within a rigid-body simulator, facilitating effective reinforcement learning for soft robot locomotion.

Findings

High-fidelity sim-to-real transfer of learned gaits on hardware

Order-of-magnitude reduction in cost of transport

Robust multi-terrain locomotion including gravel, grass, and mud

Abstract

Adaptive morphogenetic robots adapt their morphology and control policies to meet changing tasks and environmental conditions. Many such systems leverage soft components, which enable shape morphing but also introduce simulation and control challenges. Soft-body simulators remain limited in accuracy and computational tractability, while rigid-body simulators cannot capture soft-material dynamics. Here, we present a surrogate compliance modeling approach: rather than explicitly modeling soft-body physics, we introduce indirect variables representing soft-material deformation within a rigid-body simulator. We validate this approach using our amphibious robotic turtle, a quadruped with soft morphing limbs designed for multi-environment locomotion. By capturing deformation effects as changes in effective limb length and limb center of mass, and by applying reinforcement learning with extensive randomization of these indirect variables, we achieve reliable policy learning entirely in a rigid-body simulation. The resulting gaits transfer directly to hardware, demonstrating high-fidelity sim-to-real performance on hard, flat substrates and robust, though lower-fidelity, transfer on rheologically complex terrains. The learned closed-loop gaits exhibit unprecedented terrestrial maneuverability and achieve an order-of-magnitude reduction in cost of transport compared to open-loop baselines. Field experiments with the robot further demonstrate stable, multi-gait locomotion across diverse natural terrains, including gravel, grass, and mud.