A Systematic Comparison Study on Hyperparameter Optimisation of Graph Neural Networks for Molecular Property Prediction

TL;DR

This study systematically compares hyperparameter optimization methods for graph neural networks in molecular property prediction, revealing their individual strengths and guiding future research in molecular machine learning.

Contribution

It provides a theoretical and empirical comparison of TPE, CMA-ES, and random search for HPO in GNNs on small molecular datasets, highlighting their respective advantages.

Findings

Random search, TPE, and CMA-ES each excel in different molecular problems.

The study offers insights into HPO efficiency on small datasets in molecular domains.

Results suggest tailored HPO methods improve GNN performance in chemistry applications.

Abstract

Graph neural networks (GNNs) have been proposed for a wide range of graph-related learning tasks. In particular, in recent years, an increasing number of GNN systems were applied to predict molecular properties. However, a direct impediment is to select appropriate hyperparameters to achieve satisfactory performance with lower computational cost. Meanwhile, many molecular datasets are far smaller than many other datasets in typical deep learning applications. Most hyperparameter optimization (HPO) methods have not been explored in terms of their efficiencies on such small datasets in the molecular domain. In this paper, we conducted a theoretical analysis of common and specific features for two state-of-the-art and popular algorithms for HPO: TPE and CMA-ES, and we compared them with random search (RS), which is used as a baseline. Experimental studies are carried out on several benchmarks in MoleculeNet, from different perspectives to investigate the impact of RS, TPE, and CMA-ES on HPO of GNNs for molecular property prediction. In our experiments, we concluded that RS, TPE, and CMA-ES have their individual advantages in tackling different specific molecular problems. Finally, we believe our work will motivate further research on GNN as applied to molecular machine learning problems in chemistry and materials sciences.