Comparison of Optimised Geometric Deep Learning Architectures, over Varying Toxicological Assay Data Environments

TL;DR

This study systematically compares different geometric deep learning architectures, specifically GCNs, GATs, and GINs, across various toxicological datasets to identify optimal models based on data abundance and assay characteristics.

Contribution

It provides a comprehensive evaluation of GNN architectures in cheminformatics, highlighting the conditions under which each architecture performs best, which was previously underexplored.

Findings

GINs outperform GCNs and GATs in data-rich environments

GATs excel in data-scarce environments

Optimized hyperparameters reveal distinct convergence behaviors

Abstract

Geometric deep learning is an emerging technique in Artificial Intelligence (AI) driven cheminformatics, however the unique implications of different Graph Neural Network (GNN) architectures are poorly explored, for this space. This study compared performances of Graph Convolutional Networks (GCNs), Graph Attention Networks (GATs) and Graph Isomorphism Networks (GINs), applied to 7 different toxicological assay datasets of varying data abundance and endpoint, to perform binary classification of assay activation. Following pre-processing of molecular graphs, enforcement of class-balance and stratification of all datasets across 5 folds, Bayesian optimisations were carried out, for each GNN applied to each assay dataset (resulting in 21 unique Bayesian optimisations). Optimised GNNs performed at Area Under the Curve (AUC) scores ranging from 0.728-0.849 (averaged across all folds), naturally varying between specific assays and GNNs. GINs were found to consistently outperform GCNs and GATs, for the top 5 of 7 most data-abundant toxicological assays. GATs however significantly outperformed over the remaining 2 most data-scarce assays. This indicates that GINs are a more optimal architecture for data-abundant environments, whereas GATs are a more optimal architecture for data-scarce environments. Subsequent analysis of the explored higher-dimensional hyperparameter spaces, as well as optimised hyperparameter states, found that GCNs and GATs reached measurably closer optimised states with each other, compared to GINs, further indicating the unique nature of GINs as a GNN algorithm.