Building Trustworthy AI for Materials Discovery: From Autonomous Laboratories to Z-scores

TL;DR

This paper introduces the GIFTERS framework for evaluating trustworthiness in AI-driven materials discovery, emphasizing principles like interpretability, fairness, and robustness, and reviews current practices to identify gaps and future directions.

Contribution

It defines a comprehensive trustworthiness framework for AI in materials science, assesses current literature with a novel scoring system, and suggests cross-disciplinary methods to enhance trustworthiness.

Findings

Median GIFTERS score is 5/7, indicating room for improvement.

Bayesian studies often omit fair data practices.

Non-Bayesian studies frequently lack interpretability.

Abstract

Accelerated material discovery increasingly relies on artificial intelligence and machine learning, collectively termed "AI/ML". A key challenge in using AI is ensuring that human scientists trust the models are valid and reliable. Accordingly, we define a trustworthy AI framework GIFTERS for materials science and discovery to evaluate whether reported machine learning methods are generalizable, interpretable, fair, transparent, explainable, robust, and stable. Through a critical literature review, we highlight that these are the trustworthiness principles most valued by the materials discovery community. However, we also find that comprehensive approaches to trustworthiness are rarely reported; this is quantified by a median GIFTERS score of 5/7. We observe that Bayesian studies frequently omit fair data practices, while non-Bayesian studies most frequently omit interpretability. Finally, we identify approaches for improving trustworthiness methods in artificial intelligence and machine learning for materials science by considering work accomplished in other scientific disciplines such as healthcare, climate science, and natural language processing with an emphasis on methods that may transfer to materials discovery experiments. By combining these observations, we highlight the necessity of human-in-the-loop, and integrated approaches to bridge the gap between trustworthiness and uncertainty quantification for future directions of materials science research. This ensures that AI/ML methods not only accelerate discovery, but also meet ethical and scientific norms established by the materials discovery community. This work provides a road map for developing trustworthy artificial intelligence systems that will accurately and confidently enable material discovery.