Early Stopping Bayesian Optimization for Controller Tuning

TL;DR

This paper introduces an early stopping approach within Bayesian Optimization for controller tuning, significantly reducing optimization time by terminating unpromising episodes early using heuristics for partial observations.

Contribution

It proposes a novel early stopping method for Bayesian Optimization in controller tuning, handling partial observations with three heuristics, and demonstrates efficiency gains through experiments.

Findings

Early stopping reduces optimization time substantially.

Heuristics effectively incorporate partial observations.

Method performs well in both numerical and hardware experiments.

Abstract

Manual tuning of performance-critical controller parameters can be tedious and sub-optimal. Bayesian Optimization (BO) is an increasingly popular practical alternative to automatically optimize controller parameters from few experiments. Standard BO practice is to evaluate the closed-loop performance of parameters proposed during optimization on an episode with a fixed length. However, fixed-length episodes can be wasteful. For example, continuing an episode where already the start shows undesirable behavior such as strong oscillations seems pointless. Therefore, we propose a BO method that stops an episode early if suboptimality becomes apparent before an episode is completed. Such early stopping results in partial observations of the controller's performance, which cannot directly be included in standard BO. We propose three heuristics to facilitate partially observed episodes in BO. Through five numerical and one hardware experiment, we demonstrate that early stopping BO can substantially reduce the time needed for optimization.