Complex Locomotion Skill Learning via Differentiable Physics

TL;DR

This paper introduces a unified neural network controller learning framework using differentiable physics, significantly improving the complexity, diversity, and robustness of locomotion tasks for various robot designs, outperforming reinforcement learning.

Contribution

The authors develop a practical, robust framework for training neural controllers with differentiable physics, incorporating improvements like periodic activations and tailored loss functions for complex locomotion.

Findings

Outperforms reinforcement learning in convergence speed and performance

Enables interactive control of soft robot locomotion

Supports multiple goals with a single unified controller

Abstract

Differentiable physics enables efficient gradient-based optimizations of neural network (NN) controllers. However, existing work typically only delivers NN controllers with limited capability and generalizability. We present a practical learning framework that outputs unified NN controllers capable of tasks with significantly improved complexity and diversity. To systematically improve training robustness and efficiency, we investigated a suite of improvements over the baseline approach, including periodic activation functions, and tailored loss functions. In addition, we find our adoption of batching and an Adam optimizer effective in training complex locomotion tasks. We evaluate our framework on differentiable mass-spring and material point method (MPM) simulations, with challenging locomotion tasks and multiple robot designs. Experiments show that our learning framework, based on differentiable physics, delivers better results than reinforcement learning and converges much faster. We demonstrate that users can interactively control soft robot locomotion and switch among multiple goals with specified velocity, height, and direction instructions using a unified NN controller trained in our system. Code is available at https://github.com/erizmr/Complex-locomotion-skill-learning-via-differentiable-physics.