Bayesian Optimization with Structured Measurements: A Vector-Valued RKHS Framework

TL;DR

This paper extends Bayesian optimization to structured, vector-valued measurements within a vector-valued RKHS framework, enabling more informative observations and improving sample efficiency in complex systems.

Contribution

It introduces a novel Bayesian optimization approach that leverages structured vector-valued measurements and derives theoretical guarantees within a vector-valued RKHS setting.

Findings

Structured measurements improve sample efficiency in Bayesian optimization.

The proposed algorithm achieves sublinear regret rates.

Empirical results show better transfer of information across objectives.

Abstract

Bayesian optimization (BO) is an efficient framework for optimizing expensive black-box functions. However, it is typically formulated as learning an end-to-end mapping from inputs to scalar objectives, thereby discarding the potentially rich information whenever a structured system output is available. In this work, we study Bayesian optimization over a vector-valued operator with structured measurements, where each measurement observes multidimensional or functional outputs, e.g., trajectories or spatial fields, rather than a single scalar value. The objective is then defined as a linear functional of these measurements. This allows each observation to reveal substantially richer information about the underlying system compared to scalar observations. Assuming the unknown operator lies in a vector-valued reproducing kernel Hilbert space (RKHS), we derive high-probability concentration bounds for the kernel ridge regression (KRR) estimator directly in the measurement space, characterizing uncertainty in a general Hilbert space. Building on these results, we propose an algorithm based on the upper confidence bound (UCB) acquisition function with regret guarantees under mild assumptions, recovering sublinear rates for common kernels. Empirically, we demonstrate that leveraging structured measurements leads to improved sample efficiency by enabling efficient transfer of information across objectives and adaptation to time-varying settings.