DiffCoTune: Differentiable Co-Tuning for Cross-domain Robot Control

TL;DR

DiffCoTune introduces a gradient-based, automated co-tuning framework that leverages differentiable simulators to improve robot controller transfer across domains, reducing trial requirements and enhancing performance.

Contribution

It presents a novel iterative co-tuning method for both model-based and learning-based controllers using differentiable simulators, enabling systematic and scalable transfer to deployment domains.

Findings

Effective co-tuning for diverse robot tasks

Performance improvements across multiple domains

Scalable to complex controllers and tasks

Abstract

The deployment of robot controllers is hindered by modeling discrepancies due to necessary simplifications for computational tractability or inaccuracies in data-generating simulators. Such discrepancies typically require ad-hoc tuning to meet the desired performance, thereby ensuring successful transfer to a target domain. We propose a framework for automated, gradient-based tuning to enhance performance in the deployment domain by leveraging differentiable simulators. Our method collects rollouts in an iterative manner to co-tune the simulator and controller parameters, enabling systematic transfer within a few trials in the deployment domain. Specifically, we formulate multi-step objectives for tuning and employ alternating optimization to effectively adapt the controller to the deployment domain. The scalability of our framework is demonstrated by co-tuning model-based and learning-based controllers of arbitrary complexity for tasks ranging from low-dimensional cart-pole stabilization to high-dimensional quadruped and biped tracking, showing performance improvements across different deployment domains.