Introduction to the Analysis of Probabilistic Decision-Making Algorithms

TL;DR

This paper provides an accessible introduction to the theoretical analysis of probabilistic decision-making algorithms like bandits, Bayesian optimization, and tree search, emphasizing their importance in scientific discovery and cost reduction.

Contribution

It offers a self-contained, beginner-friendly overview of key algorithms and their theoretical foundations, bridging the gap for non-experts.

Findings

Clarifies the theoretical principles behind probabilistic decision algorithms.

Highlights their applications in reducing experimental costs.

Provides foundational knowledge for further research in the field.

Abstract

Decision theories offer principled methods for making choices under various types of uncertainty. Algorithms that implement these theories have been successfully applied to a wide range of real-world problems, including materials and drug discovery. Indeed, they are desirable since they can adaptively gather information to make better decisions in the future, resulting in data-efficient workflows. In scientific discovery, where experiments are costly, these algorithms can thus significantly reduce the cost of experimentation. Theoretical analyses of these algorithms are crucial for understanding their behavior and providing valuable insights for developing next-generation algorithms. However, theoretical analyses in the literature are often inaccessible to non-experts. This monograph aims to provide an accessible, self-contained introduction to the theoretical analysis of commonly used probabilistic decision-making algorithms, including bandit algorithms, Bayesian optimization, and tree search algorithms. Only basic knowledge of probability theory and statistics, along with some elementary knowledge about Gaussian processes, is assumed.