A Hierarchical Surrogate Model for Efficient Multi-Task Parameter Learning in Closed-Loop Control

TL;DR

This paper introduces a hierarchical Bayesian optimization framework that leverages structural knowledge for efficient multi-task controller parameter learning, significantly improving data efficiency and adaptability in control tasks.

Contribution

The authors develop a hierarchical Gaussian process surrogate model that exploits problem structure for transfer learning across control tasks, enhancing data efficiency in controller tuning.

Findings

Sublinear regret guarantees comparable to black-box BO.

Significant improvements in sample efficiency demonstrated in simulations.

Enhanced adaptability in multi-task control scenarios.

Abstract

Many control problems require repeated tuning and adaptation of controllers across distinct closed-loop tasks, where data efficiency and adaptability are critical. We propose a hierarchical Bayesian optimization (BO) framework that is tailored to efficient controller parameter learning in sequential decision-making and control scenarios for distinct tasks. Instead of treating the closed-loop cost as a black-box, our method exploits structural knowledge of the underlying problem, consisting of a dynamical system, a control law, and an associated closed-loop cost function. We construct a hierarchical surrogate model using Gaussian processes that capture the closed-loop state evolution under different parameterizations, while the task-specific weighting and accumulation into the closed-loop cost are computed exactly via known closed-form expressions. This allows knowledge transfer and enhanced data efficiency between different closed-loop tasks. The proposed framework retains sublinear regret guarantees on par with standard black-box BO, while enabling multi-task or transfer learning. Simulation experiments with model predictive control demonstrate substantial benefits in both sample efficiency and adaptability when compared to purely black-box BO approaches.