A Crystal-Specific Pre-Training Framework for Crystal Material Property Prediction

TL;DR

This paper introduces a novel crystal-specific pre-training framework that leverages self-supervision and periodic invariance to improve crystal property prediction, addressing label scarcity and quantum chemical principles.

Contribution

The framework uniquely incorporates periodic invariance and a mutex mask strategy, advancing crystal representation learning with limited labels.

Findings

Outperforms recent strong baselines on eight tasks.

Effectively captures periodic invariance in crystal structures.

Enhances prediction accuracy with limited labeled data.

Abstract

Crystal property prediction is a crucial aspect of developing novel materials. However, there are two technical challenges to be addressed for speeding up the investigation of crystals. First, labeling crystal properties is intrinsically difficult due to the high cost and time involved in physical simulations or lab experiments. Second, crystals adhere to a specific quantum chemical principle known as periodic invariance, which is often not captured by existing machine learning methods. To overcome these challenges, we propose the crystal-specific pre-training framework for learning crystal representations with self-supervision. The framework designs a mutex mask strategy for enhancing representation learning so as to alleviate the limited labels available for crystal property prediction. Moreover, we take into account the specific periodic invariance in crystal structures by developing a periodic invariance multi-graph module and periodic attribute learning within our framework. This framework has been tested on eight different tasks. The experimental results on these tasks show that the framework achieves promising prediction performance and is able to outperform recent strong baselines.