A Bayesian Optimization through Sequential Monte Carlo and Statistical Physics-Inspired Techniques

TL;DR

This paper introduces a novel Bayesian optimization method combining Sequential Monte Carlo and statistical physics concepts, leveraging modern ML libraries for efficient, cross-platform performance in diverse optimization tasks.

Contribution

It presents a new Bayesian optimization approach that integrates SMC and physics-inspired techniques, implemented with NumPyro and JAX for broad hardware compatibility.

Findings

Effective optimization across CPUs, GPUs, and TPUs.

High performance with low entry barrier for users.

Potential for wide application in various fields.

Abstract

In this paper, we propose an approach for an application of Bayesian optimization using Sequential Monte Carlo (SMC) and concepts from the statistical physics of classical systems. Our method leverages the power of modern machine learning libraries such as NumPyro and JAX, allowing us to perform Bayesian optimization on multiple platforms, including CPUs, GPUs, TPUs, and in parallel. Our approach enables a low entry level for exploration of the methods while maintaining high performance. We present a promising direction for developing more efficient and effective techniques for a wide range of optimization problems in diverse fields.