Scalable training of graph convolutional neural networks for fast and accurate predictions of HOMO-LUMO gap in molecules

TL;DR

This paper presents a scalable approach to training graph convolutional neural networks on high-performance computing systems, enabling fast and accurate prediction of the HOMO-LUMO gap in molecules using large datasets and distributed training techniques.

Contribution

The work introduces HydraGNN and ADIOS frameworks for efficient large-scale GCNN training on HPC systems, achieving significant speedups and linear scaling across thousands of GPUs.

Findings

Data loading time reduced by up to 4.2 times

Achieved linear scaling up to 1,024 GPUs

Demonstrated effective prediction of HOMO-LUMO gaps in large molecular datasets

Abstract

Graph Convolutional Neural Network (GCNN) is a popular class of deep learning (DL) models in material science to predict material properties from the graph representation of molecular structures. Training an accurate and comprehensive GCNN surrogate for molecular design requires large-scale graph datasets and is usually a time-consuming process. Recent advances in GPUs and distributed computing open a path to reduce the computational cost for GCNN training effectively. However, efficient utilization of high performance computing (HPC) resources for training requires simultaneously optimizing large-scale data management and scalable stochastic batched optimization techniques. In this work, we focus on building GCNN models on HPC systems to predict material properties of millions of molecules. We use HydraGNN, our in-house library for large-scale GCNN training, leveraging distributed data parallelism in PyTorch. We use ADIOS, a high-performance data management framework for efficient storage and reading of large molecular graph data. We perform parallel training on two open-source large-scale graph datasets to build a GCNN predictor for an important quantum property known as the HOMO-LUMO gap. We measure the scalability, accuracy, and convergence of our approach on two DOE supercomputers: the Summit supercomputer at the Oak Ridge Leadership Computing Facility (OLCF) and the Perlmutter system at the National Energy Research Scientific Computing Center (NERSC). We present our experimental results with HydraGNN showing i) reduction of data loading time up to 4.2 times compared with a conventional method and ii) linear scaling performance for training up to 1,024 GPUs on both Summit and Perlmutter.