Doubly Bayesian Optimization

TL;DR

This paper introduces a probabilistic programming approach to Bayesian optimization that effectively handles complex domains, incorporates domain knowledge, and simplifies advanced techniques, demonstrated on benchmarks and drug development.

Contribution

It presents a novel probabilistic program embedding of BO that addresses domain challenges and simplifies implementation of advanced methods.

Findings

Effective on optimization benchmarks

Handles noisy, non-smooth, high-dimensional domains

Proven useful in drug development scenario

Abstract

Probabilistic programming systems enable users to encode model structure and naturally reason about uncertainties, which can be leveraged towards improved Bayesian optimization (BO) methods. Here we present a probabilistic program embedding of BO that is capable of addressing main issues such as problematic domains (noisy, non-smooth, high-dimensional) and the neglected inner-optimization. Not only can we utilize programmable structure to incorporate domain knowledge to aid optimization, but dealing with uncertainties and implementing advanced BO techniques become trivial, crucial for use in practice (particularly for non-experts). We demonstrate the efficacy of the approach on optimization benchmarks and a real-world drug development scenario.