Crystal Twins: Self-supervised Learning for Crystalline Material Property Prediction

TL;DR

This paper introduces Crystal Twins, a self-supervised learning approach using graph neural networks to predict properties of crystalline materials, reducing the need for large labeled datasets and improving benchmark performance.

Contribution

It presents a novel SSL framework for crystalline materials that pre-trains GNNs on unlabeled data, enhancing property prediction accuracy.

Findings

Significant performance improvements on 7 property prediction benchmarks.

Effective use of unlabeled data reduces reliance on costly labeled datasets.

Pre-training with SSL enhances GNN generalization for material properties.

Abstract

Machine learning (ML) models have been widely successful in the prediction of material properties. However, large labeled datasets required for training accurate ML models are elusive and computationally expensive to generate. Recent advances in Self-Supervised Learning (SSL) frameworks capable of training ML models on unlabeled data have mitigated this problem and demonstrated superior performance in computer vision and natural language processing tasks. Drawing inspiration from the developments in SSL, we introduce Crystal Twins (CT): an SSL method for crystalline materials property prediction. Using a large unlabeled dataset, we pre-train a Graph Neural Network (GNN) by applying the redundancy reduction principle to the graph latent embeddings of augmented instances obtained from the same crystalline system. By sharing the pre-trained weights when fine-tuning the GNN for regression tasks, we significantly improve the performance for 7 challenging material property prediction benchmarks