Assessing the Frontier: Active Learning, Model Accuracy, and Multi-objective Materials Discovery and Optimization

TL;DR

This paper introduces a new error metric called Pareto shell-scope error to better evaluate machine learning models for materials discovery, providing insights into active learning performance and guiding acquisition function design.

Contribution

It proposes the Pareto shell-scope error metric and investigates its relation to active learning success in materials discovery, offering new diagnostic tools.

Findings

Pareto shell-scope error better predicts discovery success.

Acquisition function fidelity impacts active learning performance.

New insights for designing effective acquisition functions.

Abstract

Discovering novel materials can be greatly accelerated by iterative machine learning-informed proposal of candidates---active learning. However, standard \emph{global-scope error} metrics for model quality are not predictive of discovery performance, and can be misleading. We introduce the notion of \emph{Pareto shell-scope error} to help judge the suitability of a model for proposing material candidates. Further, through synthetic cases and a thermoelectric dataset, we probe the relation between acquisition function fidelity and active learning performance. Results suggest novel diagnostic tools, as well as new insights for acquisition function design.