Molecular Contrastive Learning of Representations via Graph Neural Networks

TL;DR

MolCLR introduces a self-supervised contrastive learning framework using graph neural networks to improve molecular property prediction, leveraging large unlabeled datasets to enhance generalization and achieve state-of-the-art results.

Contribution

This work presents a novel contrastive learning approach for GNNs on molecular graphs, utilizing new augmentation techniques and large unlabeled data for improved molecular representations.

Findings

Significantly improves GNN performance on molecular benchmarks.

Achieves state-of-the-art results after fine-tuning.

Learns chemically meaningful molecular embeddings.

Abstract

Molecular Machine Learning (ML) bears promise for efficient molecule property prediction and drug discovery. However, labeled molecule data can be expensive and time-consuming to acquire. Due to the limited labeled data, it is a great challenge for supervised-learning ML models to generalize to the giant chemical space. In this work, we present MolCLR: Molecular Contrastive Learning of Representations via Graph Neural Networks (GNNs), a self-supervised learning framework that leverages large unlabeled data (~10M unique molecules). In MolCLR pre-training, we build molecule graphs and develop GNN encoders to learn differentiable representations. Three molecule graph augmentations are proposed: atom masking, bond deletion, and subgraph removal. A contrastive estimator maximizes the agreement of augmentations from the same molecule while minimizing the agreement of different molecules. Experiments show that our contrastive learning framework significantly improves the performance of GNNs on various molecular property benchmarks including both classification and regression tasks. Benefiting from pre-training on the large unlabeled database, MolCLR even achieves state-of-the-art on several challenging benchmarks after fine-tuning. Additionally, further investigations demonstrate that MolCLR learns to embed molecules into representations that can distinguish chemically reasonable molecular similarities.