Achieving Precise and Reliable Locomotion with Differentiable Simulation-Based System Identification

TL;DR

This paper introduces a differentiable simulation-based system identification method that estimates physical parameters of robots using trajectory data, improving the accuracy and reliability of bipedal locomotion control.

Contribution

The paper presents a novel framework integrating system identification into reinforcement learning with differentiable simulation, enabling scalable and flexible parameter estimation using only trajectory data.

Findings

Significantly improves trajectory following accuracy.

Supports estimation of physical properties like mass and inertia.

Handles complex nonlinear behaviors with neural network approximations.

Abstract

Accurate system identification is crucial for reducing trajectory drift in bipedal locomotion, particularly in reinforcement learning and model-based control. In this paper, we present a novel control framework that integrates system identification into the reinforcement learning training loop using differentiable simulation. Unlike traditional approaches that rely on direct torque measurements, our method estimates system parameters using only trajectory data (positions, velocities) and control inputs. We leverage the differentiable simulator MuJoCo-XLA to optimize system parameters, ensuring that simulated robot behavior closely aligns with real-world motion. This framework enables scalable and flexible parameter optimization. Accurate system identification is crucial for reducing trajectory drift in bipedal locomotion, particularly in reinforcement learning and model-based control. In this paper, we present a novel control framework that integrates system identification into the reinforcement learning training loop using differentiable simulation. Unlike traditional approaches that rely on direct torque measurements, our method estimates system parameters using only trajectory data (positions, velocities) and control inputs. We leverage the differentiable simulator MuJoCo-XLA to optimize system parameters, ensuring that simulated robot behavior closely aligns with real-world motion. This framework enables scalable and flexible parameter optimization. It supports fundamental physical properties such as mass and inertia. Additionally, it handles complex system nonlinear behaviors, including advanced friction models, through neural network approximations. Experimental results show that our framework significantly improves trajectory following.