OCCAM: Online Continuous Controller Adaptation with Meta-Learned Models

TL;DR

OCCAM introduces a meta-learning based framework for online control adaptation that quickly adjusts robot controllers across diverse environments by combining predictive modeling with Bayesian estimation.

Contribution

The paper presents a novel framework integrating meta-learning and Bayesian estimation for rapid online controller adaptation under domain shifts.

Findings

Successfully adapted controllers for four diverse robotic systems

Achieved rapid online adaptation with minimal data

Demonstrated effectiveness in both simulated and real-world scenarios

Abstract

Control tuning and adaptation present a significant challenge to the usage of robots in diverse environments. It is often nontrivial to find a single set of control parameters by hand that work well across the broad array of environments and conditions that a robot might encounter. Automated adaptation approaches must utilize prior knowledge about the system while adapting to significant domain shifts to find new control parameters quickly. In this work, we present a general framework for online controller adaptation that deals with these challenges. We combine meta-learning with Bayesian recursive estimation to learn prior predictive models of system performance that quickly adapt to online data, even when there is significant domain shift. These predictive models can be used as cost functions within efficient sampling-based optimization routines to find new control parameters online that maximize system performance. Our framework is powerful and flexible enough to adapt controllers for four diverse systems: a simulated race car, a simulated quadrupedal robot, and a simulated and physical quadrotor. The video and code can be found at https://hersh500.github.io/occam.