A Continuous Relaxation for Discrete Bayesian Optimization

TL;DR

This paper introduces a continuous relaxation method for discrete Bayesian optimization, enabling efficient inference and optimization with limited data, especially in bio-chemical sequence tasks, by incorporating prior knowledge through a novel covariance function.

Contribution

The paper presents a novel continuous relaxation approach for discrete Bayesian optimization that allows direct incorporation of prior knowledge and efficient optimization in data-scarce settings.

Findings

Effective optimization of bio-chemical sequences demonstrated

Method outperforms traditional discrete approaches in limited data scenarios

Flexible optimization using both continuous and discrete algorithms

Abstract

To optimize efficiently over discrete data and with only few available target observations is a challenge in Bayesian optimization. We propose a continuous relaxation of the objective function and show that inference and optimization can be computationally tractable. We consider in particular the optimization domain where very few observations and strict budgets exist; motivated by optimizing protein sequences for expensive to evaluate bio-chemical properties. The advantages of our approach are two-fold: the problem is treated in the continuous setting, and available prior knowledge over sequences can be incorporated directly. More specifically, we utilize available and learned distributions over the problem domain for a weighting of the Hellinger distance which yields a covariance function. We show that the resulting acquisition function can be optimized with both continuous or discrete optimization algorithms and empirically assess our method on two bio-chemical sequence optimization tasks.