Practical Bayesian Optimization for Model Fitting with Bayesian Adaptive Direct Search

TL;DR

This paper introduces BADS, a hybrid Bayesian optimization algorithm tailored for fitting complex, noisy models in neuroscience, demonstrating its competitive performance against existing nonconvex optimizers on real-world problems.

Contribution

The paper presents BADS, a novel hybrid Bayesian optimization method that efficiently handles noisy, complex model fitting tasks, outperforming many existing optimizers in neuroscience applications.

Findings

BADS achieves competitive or superior results compared to state-of-the-art optimizers.

BADS demonstrates robustness and efficiency on real neuroscience data.

Default settings of BADS consistently find high-quality solutions.

Abstract

Computational models in fields such as computational neuroscience are often evaluated via stochastic simulation or numerical approximation. Fitting these models implies a difficult optimization problem over complex, possibly noisy parameter landscapes. Bayesian optimization (BO) has been successfully applied to solving expensive black-box problems in engineering and machine learning. Here we explore whether BO can be applied as a general tool for model fitting. First, we present a novel hybrid BO algorithm, Bayesian adaptive direct search (BADS), that achieves competitive performance with an affordable computational overhead for the running time of typical models. We then perform an extensive benchmark of BADS vs. many common and state-of-the-art nonconvex, derivative-free optimizers, on a set of model-fitting problems with real data and models from six studies in behavioral, cognitive, and computational neuroscience. With default settings, BADS consistently finds comparable or better solutions than other methods, including `vanilla' BO, showing great promise for advanced BO techniques, and BADS in particular, as a general model-fitting tool.