On the Design of LQR Kernels for Efficient Controller Learning

TL;DR

This paper introduces LQR-specific kernels for Bayesian optimization to improve controller learning efficiency in nonlinear systems, demonstrating superior performance over standard kernels in simulations.

Contribution

It develops novel LQR-based kernels that incorporate system structure, enhancing Bayesian optimization for control tasks.

Findings

LQR kernels outperform standard kernels in learning speed.

Simulations show improved control performance with LQR kernels.

LQR kernels effectively leverage system structure for better learning.

Abstract

Finding optimal feedback controllers for nonlinear dynamic systems from data is hard. Recently, Bayesian optimization (BO) has been proposed as a powerful framework for direct controller tuning from experimental trials. For selecting the next query point and finding the global optimum, BO relies on a probabilistic description of the latent objective function, typically a Gaussian process (GP). As is shown herein, GPs with a common kernel choice can, however, lead to poor learning outcomes on standard quadratic control problems. For a first-order system, we construct two kernels that specifically leverage the structure of the well-known Linear Quadratic Regulator (LQR), yet retain the flexibility of Bayesian nonparametric learning. Simulations of uncertain linear and nonlinear systems demonstrate that the LQR kernels yield superior learning performance.