Attention-Based Explainability for Structure-Property Relationships

TL;DR

This paper explores the use of attention-based neural networks to interpret structure-property relationships in materials science, specifically revealing physical mechanisms behind ferroelectric properties in PbTiO3 thin films.

Contribution

It demonstrates the effectiveness of attention mechanisms in neural networks for physically interpretable insights into structure-property relationships in materials science.

Findings

Attention scores reveal influence of domain patterns on polarization switching.

Attention-based models outperform classical interpretability methods like SHAP in highlighting structural features.

Attention mechanisms efficiently identify meaningful physical correlations in materials data.

Abstract

Machine learning methods are emerging as a universal paradigm for constructing correlative structure-property relationships in materials science based on multimodal characterization. However, this necessitates development of methods for physical interpretability of the resulting correlative models. Here, we demonstrate the potential of attention-based neural networks for revealing structure-property relationships and the underlying physical mechanisms, using the ferroelectric properties of PbTiO3 thin films as a case study. Through the analysis of attention scores, we disentangle the influence of distinct domain patterns on the polarization switching process. The attention-based Transformer model is explored both as a direct interpretability tool and as a surrogate for explaining representations learned via unsupervised machine learning, enabling the identification of physically grounded correlations. We compare attention-derived interpretability scores with classical SHapley Additive exPlanations (SHAP) analysis and show that, in contrast to applications in natural language processing, attention mechanisms in materials science exhibit high efficiency in highlighting meaningful structural features.