Bayesian Optimization Using Domain Knowledge on the ATRIAS Biped

TL;DR

This paper enhances Bayesian Optimization for robotic controllers by incorporating domain knowledge through a feature transform, improving sample efficiency and transferability from simulation to real hardware, specifically applied to the ATRIAS biped.

Contribution

It introduces a generalized feature transform that leverages domain knowledge to reduce dimensionality, improving BO performance for diverse robot morphologies including ATRIAS.

Findings

Feature transform accelerates learning on hardware and simulation.

Improved sample efficiency over traditional Bayesian Optimization.

Effective on multiple walking controllers for ATRIAS robot.

Abstract

Controllers in robotics often consist of expert-designed heuristics, which can be hard to tune in higher dimensions. It is typical to use simulation to learn these parameters, but controllers learned in simulation often don't transfer to hardware. This necessitates optimization directly on hardware. However, collecting data on hardware can be expensive. This has led to a recent interest in adapting data-efficient learning techniques to robotics. One popular method is Bayesian Optimization (BO), a sample-efficient black-box optimization scheme, but its performance typically degrades in higher dimensions. We aim to overcome this problem by incorporating domain knowledge to reduce dimensionality in a meaningful way, with a focus on bipedal locomotion. In previous work, we proposed a transformation based on knowledge of human walking that projected a 16-dimensional controller to a 1-dimensional space. In simulation, this showed enhanced sample efficiency when optimizing human-inspired neuromuscular walking controllers on a humanoid model. In this paper, we present a generalized feature transform applicable to non-humanoid robot morphologies and evaluate it on the ATRIAS bipedal robot -- in simulation and on hardware. We present three different walking controllers; two are evaluated on the real robot. Our results show that this feature transform captures important aspects of walking and accelerates learning on hardware and simulation, as compared to traditional BO.