High-Dimensional Surrogate Modeling for Closed-Loop Learning of Neural-Network-Parameterized Model Predictive Control

TL;DR

This paper demonstrates that Bayesian neural networks can effectively serve as surrogate models in Bayesian optimization for high-dimensional controller parameter tuning, outperforming Gaussian processes especially in very high-dimensional spaces.

Contribution

It introduces the use of Bayesian neural networks as surrogate models for high-dimensional controller optimization, showing improved convergence and scalability over traditional Gaussian processes.

Findings

Bayesian neural networks outperform Gaussian processes in high-dimensional settings.

Infinite-width Bayesian neural networks maintain performance with over 1000 parameters.

Surrogate models significantly improve closed-loop cost optimization efficiency.

Abstract

Learning controller parameters from closed-loop data has been shown to improve closed-loop performance. Bayesian optimization, a widely used black-box and sample-efficient learning method, constructs a probabilistic surrogate of the closed-loop performance from few experiments and uses it to select informative controller parameters. However, it typically struggles with dense high-dimensional controller parameterizations, as they may appear, for example, in tuning model predictive controllers, because standard surrogate models fail to capture the structure of such spaces. This work suggests that the use of Bayesian neural networks as surrogate models may help to mitigate this limitation. Through a comparison between Gaussian processes with Matern kernels, finite-width Bayesian neural networks, and infinite-width Bayesian neural networks on a cart-pole task, we find that Bayesian neural network surrogate models achieve faster and more reliable convergence of the closed-loop cost and enable successful optimization of parameterizations with hundreds of dimensions. Infinite-width Bayesian neural networks also maintain performance in settings with more than one thousand parameters, whereas Matern-kernel Gaussian processes rapidly lose effectiveness. These results indicate that Bayesian neural network surrogate models may be suitable for learning dense high-dimensional controller parameterizations and offer practical guidance for selecting surrogate models in learning-based controller design.