Leveraging Uncertainty from Deep Learning for Trustworthy Materials Discovery Workflows

TL;DR

This paper demonstrates how predictive uncertainty from deep neural networks can enhance materials discovery workflows by determining training data needs, detecting confusing samples, and identifying out-of-distribution data, thereby improving model reliability.

Contribution

It introduces methods to utilize uncertainty estimates for data requirement assessment, decision referral, and out-of-distribution detection in materials science applications.

Findings

Uncertainty guides optimal training data size.

Uncertainty-based decision referral improves model reliability.

Out-of-distribution detection accurately identifies data shifts.

Abstract

In this paper, we leverage predictive uncertainty of deep neural networks to answer challenging questions material scientists usually encounter in machine learning based materials applications workflows. First, we show that by leveraging predictive uncertainty, a user can determine the required training data set size necessary to achieve a certain classification accuracy. Next, we propose uncertainty guided decision referral to detect and refrain from making decisions on confusing samples. Finally, we show that predictive uncertainty can also be used to detect out-of-distribution test samples. We find that this scheme is accurate enough to detect a wide range of real-world shifts in data, e.g., changes in the image acquisition conditions or changes in the synthesis conditions. Using microstructure information from scanning electron microscope (SEM) images as an example use case, we show that leveraging uncertainty-aware deep learning can significantly improve the performance and dependability of classification models.