Synergistic Fusion of Graph and Transformer Features for Enhanced Molecular Property Prediction

TL;DR

SYN-FUSION is a novel method that combines GNN and Transformer features to improve molecular property prediction by capturing both global structure and atom-level details, outperforming previous models.

Contribution

The paper introduces SYN-FUSION, a new approach that synergistically fuses pre-trained GNN and Transformer features for enhanced molecular property prediction.

Findings

Outperforms previous models on most MoleculeNet datasets

Achieves comparable performance with joint GNN-Transformer models

Demonstrates the effectiveness of feature fusion through ablation studies

Abstract

Molecular property prediction is a critical task in computational drug discovery. While recent advances in Graph Neural Networks (GNNs) and Transformers have shown to be effective and promising, they face the following limitations: Transformer self-attention does not explicitly consider the underlying molecule structure while GNN feature representation alone is not sufficient to capture granular and hidden interactions and characteristics that distinguish similar molecules. To address these limitations, we propose SYN- FUSION, a novel approach that synergistically combines pre-trained features from GNNs and Transformers. This approach provides a comprehensive molecular representation, capturing both the global molecule structure and the individual atom characteristics. Experimental results on MoleculeNet benchmarks demonstrate superior performance, surpassing previous models in 5 out of 7 classification datasets and 4 out of 6 regression datasets. The performance of SYN-FUSION has been compared with other Graph-Transformer models that have been jointly trained using a combination of transformer and graph features, and it is found that our approach is on par with those models in terms of performance. Extensive analysis of the learned fusion model across aspects such as loss, latent space, and weight distribution further validates the effectiveness of SYN-FUSION. Finally, an ablation study unequivocally demonstrates that the synergy achieved by SYN-FUSION surpasses the performance of its individual model components and their ensemble, offering a substantial improvement in predicting molecular properties.