DiffSim2Real: Deploying Quadrupedal Locomotion Policies Purely Trained in Differentiable Simulation

TL;DR

This paper demonstrates that quadrupedal locomotion policies trained solely in a differentiable simulator can be successfully transferred to real robots, thanks to a smooth contact model balancing gradient informativeness and physical accuracy.

Contribution

First successful transfer of quadrupedal locomotion policies trained exclusively in differentiable simulation to real-world robots.

Findings

Policies trained in differentiable simulation can be deployed on real robots.

A smooth contact model enables effective sim-to-real transfer.

First demonstration of purely simulation-trained quadrupedal locomotion in real world.

Abstract

Differentiable simulators provide analytic gradients, enabling more sample-efficient learning algorithms and paving the way for data intensive learning tasks such as learning from images. In this work, we demonstrate that locomotion policies trained with analytic gradients from a differentiable simulator can be successfully transferred to the real world. Typically, simulators that offer informative gradients lack the physical accuracy needed for sim-to-real transfer, and vice-versa. A key factor in our success is a smooth contact model that combines informative gradients with physical accuracy, ensuring effective transfer of learned behaviors. To the best of our knowledge, this is the first time a real quadrupedal robot is able to locomote after training exclusively in a differentiable simulation.