PyBADS: Fast and robust black-box optimization in Python

TL;DR

PyBADS is a Python implementation of the BADS algorithm, enabling fast, robust black-box optimization for complex, noisy, and expensive functions, especially useful in computational modeling and neuroscience.

Contribution

This paper introduces PyBADS, a user-friendly Python package that implements BADS, offering efficient optimization for challenging black-box problems with extensive benchmarking.

Findings

Performs on par or better than state-of-the-art optimizers

Scales efficiently to 20 parameters with bounds or no constraints

Effective for model fitting in neuroscience and cognitive science

Abstract

PyBADS is a Python implementation of the Bayesian Adaptive Direct Search (BADS) algorithm for fast and robust black-box optimization (Acerbi and Ma 2017). BADS is an optimization algorithm designed to efficiently solve difficult optimization problems where the objective function is rough (non-convex, non-smooth), mildly expensive (e.g., the function evaluation requires more than 0.1 seconds), possibly noisy, and gradient information is unavailable. With BADS, these issues are well addressed, making it an excellent choice for fitting computational models using methods such as maximum-likelihood estimation. The algorithm scales efficiently to black-box functions with up to $D \approx 20$ continuous input parameters and supports bounds or no constraints. PyBADS comes along with an easy-to-use Pythonic interface for running the algorithm and inspecting its results. PyBADS only requires the user to provide a Python function for evaluating the target function, and optionally other constraints. Extensive benchmarks on both artificial test problems and large real model-fitting problems models drawn from cognitive, behavioral and computational neuroscience, show that BADS performs on par with or better than many other common and state-of-the-art optimizers (Acerbi and Ma 2017), making it a general model-fitting tool which provides fast and robust solutions.