High Entropy Alloy property predictions using Transformer-based language model

TL;DR

This paper presents a transformer-based machine learning model that predicts mechanical properties of high-entropy alloys by leveraging transfer learning and attention mechanisms, improving accuracy and interpretability over traditional models.

Contribution

It introduces a novel transformer-based approach for HEA property prediction, utilizing transfer learning to address data scarcity and providing insights through attention visualization.

Findings

Transformer model outperforms traditional regression models in accuracy.

Transfer learning enhances prediction quality with limited HEA data.

Attention weights reveal meaningful elemental interactions.

Abstract

This study introduces a language transformer-based machine learning model to predict key mechanical properties of high-entropy alloys (HEAs), addressing the challenges due to their complex, multi-principal element compositions and limited experimental data. By pre-training the transformer on extensive synthetic materials data and fine-tuning it with specific HEA datasets, the model effectively captures intricate elemental interactions through self-attention mechanisms. This approach mitigates data scarcity issues via transfer learning, enhancing predictive accuracy for properties like elongation (%) and ultimate tensile strength (UTS) compared to traditional regression models such as Random Forests and Gaussian Processes. The model's interpretability is enhanced by visualizing attention weights, revealing significant elemental relationships that align with known metallurgical principles. This work demonstrates the potential of transformer models to accelerate materials discovery and optimization, enabling accurate property predictions, thereby advancing the field of materials informatics.