S2SNet: A Pretrained Neural Network for Superconductivity Discovery

TL;DR

This paper introduces S2SNet, a novel pretrained neural network utilizing attention mechanisms and a new dataset to predict superconductivity based solely on crystal structures, achieving state-of-the-art accuracy.

Contribution

The work presents the first model to predict superconductivity using only crystal structure data, leveraging a new dataset and pretraining techniques.

Findings

Achieved 92% out-of-sample accuracy

Attained 0.92 AUC in superconductivity prediction

First to predict superconductivity from crystal structures alone

Abstract

Superconductivity allows electrical current to flow without any energy loss, and thus making solids superconducting is a grand goal of physics, material science, and electrical engineering. More than 16 Nobel Laureates have been awarded for their contribution to superconductivity research. Superconductors are valuable for sustainable development goals (SDGs), such as climate change mitigation, affordable and clean energy, industry, innovation and infrastructure, and so on. However, a unified physics theory explaining all superconductivity mechanism is still unknown. It is believed that superconductivity is microscopically due to not only molecular compositions but also the geometric crystal structure. Hence a new dataset, S2S, containing both crystal structures and superconducting critical temperature, is built upon SuperCon and Material Project. Based on this new dataset, we propose a novel model, S2SNet, which utilizes the attention mechanism for superconductivity prediction. To overcome the shortage of data, S2SNet is pre-trained on the whole Material Project dataset with Masked-Language Modeling (MLM). S2SNet makes a new state-of-the-art, with out-of-sample accuracy of 92% and Area Under Curve (AUC) of 0.92. To the best of our knowledge, S2SNet is the first work to predict superconductivity with only information of crystal structures. This work is beneficial to superconductivity discovery and further SDGs. Code and datasets are available in https://github.com/zjuKeLiu/S2SNet