Deep Kernels for Optimizing Locomotion Controllers

TL;DR

This paper introduces a method to automatically learn distance metrics using neural networks to improve the sample efficiency of Bayesian Optimization in tuning locomotion controllers for robots, demonstrated on real hardware and simulations.

Contribution

The paper presents a neural network-based approach to learn distance metrics for Bayesian Optimization, enhancing sample efficiency in optimizing high-dimensional locomotion controllers.

Findings

Improved sample efficiency on ATRIAS robot hardware.

Significant optimization results in simulated 7-link robot model.

Effective in perturbed environments, demonstrating robustness.

Abstract

Sample efficiency is important when optimizing parameters of locomotion controllers, since hardware experiments are time consuming and expensive. Bayesian Optimization, a sample-efficient optimization framework, has recently been widely applied to address this problem, but further improvements in sample efficiency are needed for practical applicability to real-world robots and high-dimensional controllers. To address this, prior work has proposed using domain expertise for constructing custom distance metrics for locomotion. In this work we show how to learn such a distance metric automatically. We use a neural network to learn an informed distance metric from data obtained in high-fidelity simulations. We conduct experiments on two different controllers and robot architectures. First, we demonstrate improvement in sample efficiency when optimizing a 5-dimensional controller on the ATRIAS robot hardware. We then conduct simulation experiments to optimize a 16-dimensional controller for a 7-link robot model and obtain significant improvements even when optimizing in perturbed environments. This demonstrates that our approach is able to enhance sample efficiency for two different controllers, hence is a fitting candidate for further experiments on hardware in the future.