Analogical-based Bayesian Optimization

TL;DR

This paper introduces Analogical-based Bayesian Optimization, a novel approach that maximizes black-box functions over complex domains using only similarity scores, extending traditional Bayesian Optimization to non-vectorial objects.

Contribution

It proposes a new framework that replaces kernel similarity with a genetic similarity score within Gaussian Processes for optimization over non-vectorial domains.

Findings

Defines influence level for Gaussian Processes to replace kernel similarity

Develops strategies for batch query selection in high-dimensional spaces

Extends Bayesian Optimization to domains with only similarity scores

Abstract

Some real-world problems revolve to solve the optimization problem \max_{x\in\mathcal{X}}f\left(x\right) where f\left(.\right) is a black-box function and X might be the set of non-vectorial objects (e.g., distributions) where we can only define a symmetric and non-negative similarity score on it. This setting requires a novel view for the standard framework of Bayesian Optimization that generalizes the core insightful spirit of this framework. With this spirit, in this paper, we propose Analogical-based Bayesian Optimization that can maximize black-box function over a domain where only a similarity score can be defined. Our pathway is as follows: we first base on the geometric view of Gaussian Processes (GP) to define the concept of influence level that allows us to analytically represent predictive means and variances of GP posteriors and base on that view to enable replacing kernel similarity by a more genetic similarity score. Furthermore, we also propose two strategies to find a batch of query points that can efficiently handle high dimensional data.