Predicting Many Crystal Properties via an Adaptive Transformer-based Framework

TL;DR

This paper introduces CrystalBERT, a transformer-based framework that effectively predicts various crystal properties by integrating diverse features, achieving high accuracy and interpretability in materials science applications.

Contribution

The paper presents a novel adaptive transformer model, CrystalBERT, that combines multiple features for accurate and interpretable prediction of diverse crystal properties.

Findings

Achieves 91% accuracy in topological classification.

Space group and elemental features are crucial for predictions.

Surpasses prior models in predicting complex properties.

Abstract

Machine learning has revolutionized many fields, including materials science. However, predicting properties of crystalline materials using machine learning faces challenges in input encoding, output versatility, and interpretability. We introduce CrystalBERT, an adaptable transformer-based framework integrating space group, elemental, and unit cell information. This novel structure can seamlessly combine diverse features and accurately predict various physical properties, including topological properties, superconducting transition temperatures, dielectric constants, and more. CrystalBERT provides insightful interpretations of features influencing target properties. Our results indicate that space group and elemental information are crucial for predicting topological and superconducting properties, underscoring their intricate nature. By incorporating these features, we achieve 91\% accuracy in topological classification, surpassing prior studies and identifying previously misclassified materials. This research demonstrates that integrating diverse material information enhances the prediction of complex material properties, paving the way for more accurate and interpretable machine learning models in materials science.