Metrics for Benchmarking and Uncertainty Quantification: Quality, Applicability, and a Path to Best Practices for Machine Learning in Chemistry

TL;DR

This review emphasizes the importance of proper metrics for benchmarking and quantifying uncertainty in machine learning models for chemistry, highlighting their role in model validation, comparison, and establishing best practices.

Contribution

It discusses the current state and challenges of statistical metrics and uncertainty quantification in chemical machine learning, proposing a pathway toward standardized best practices.

Findings

Metrics are often overlooked in chemical ML validation

Uncertainty quantification enhances model reliability and applicability

Guidelines for best practices are proposed

Abstract

This review aims to draw attention to two issues of concern when we set out to make machine learning work in the chemical and materials domain, i.e., statistical loss function metrics for the validation and benchmarking of data-derived models, and the uncertainty quantification of predictions made by them. They are often overlooked or underappreciated topics as chemists typically only have limited training in statistics. Aside from helping to assess the quality, reliability, and applicability of a given model, these metrics are also key to comparing the performance of different models and thus for developing guidelines and best practices for the successful application of machine learning in chemistry.