Uncertainty-Aware Mixed-Variable Machine Learning for Materials Design

TL;DR

This paper compares frequentist and Bayesian uncertainty quantification methods in Bayesian Optimization for mixed-variable materials design, providing insights and guidance for selecting models based on problem complexity.

Contribution

It systematically evaluates and compares frequentist and Bayesian approaches for uncertainty quantification in mixed-variable Bayesian Optimization for materials discovery.

Findings

Bayesian models perform better on complex problems.

Frequentist models are more efficient on simpler problems.

Performance differences depend on problem dimensionality and complexity.

Abstract

Data-driven design shows the promise of accelerating materials discovery but is challenging due to the prohibitive cost of searching the vast design space of chemistry, structure, and synthesis methods. Bayesian Optimization (BO) employs uncertainty-aware machine learning models to select promising designs to evaluate, hence reducing the cost. However, BO with mixed numerical and categorical variables, which is of particular interest in materials design, has not been well studied. In this work, we survey frequentist and Bayesian approaches to uncertainty quantification of machine learning with mixed variables. We then conduct a systematic comparative study of their performances in BO using a popular representative model from each group, the random forest-based Lolo model (frequentist) and the latent variable Gaussian process model (Bayesian). We examine the efficacy of the two models in the optimization of mathematical functions, as well as properties of structural and functional materials, where we observe performance differences as related to problem dimensionality and complexity. By investigating the machine learning models' predictive and uncertainty estimation capabilities, we provide interpretations of the observed performance differences. Our results provide practical guidance on choosing between frequentist and Bayesian uncertainty-aware machine learning models for mixed-variable BO in materials design.