Function-on-Function Bayesian Optimization

TL;DR

This paper introduces a novel Bayesian optimization framework for optimizing objectives where both inputs and outputs are functions, using a new Gaussian process model and scalable algorithms, with proven theoretical properties and superior experimental results.

Contribution

The paper proposes a new function-on-function Gaussian process model and an associated Bayesian optimization framework, filling a gap in optimizing function-valued objectives.

Findings

Superior performance on synthetic data

Effective in real-world applications

Theoretically grounded approach

Abstract

Bayesian optimization (BO) has been widely used to optimize expensive and gradient-free objective functions across various domains. However, existing BO methods have not addressed the objective where both inputs and outputs are functions, which increasingly arise in complex systems as advanced sensing technologies. To fill this gap, we propose a novel function-on-function Bayesian optimization (FFBO) framework. Specifically, we first introduce a function-on-function Gaussian process (FFGP) model with a separable operator-valued kernel to capture the correlations between function-valued inputs and outputs. Compared to existing Gaussian process models, FFGP is modeled directly in the function space. Based on FFGP, we define a scalar upper confidence bound (UCB) acquisition function using a weighted operator-based scalarization strategy. Then, a scalable functional gradient ascent algorithm (FGA) is developed to efficiently identify the optimal function-valued input. We further analyze the theoretical properties of the proposed method. Extensive experiments on synthetic and real-world data demonstrate the superior performance of FFBO over existing approaches.