Dual-Modality Representation Learning for Molecular Property Prediction

TL;DR

This paper introduces a dual-modality learning approach that combines graph and SMILES representations of molecules using cross-attention, significantly improving drug property prediction accuracy across multiple datasets.

Contribution

The paper proposes the DMCA method, a novel cross-attention framework that effectively fuses graph and SMILES representations for enhanced molecular property prediction.

Findings

DMCA achieves state-of-the-art performance on eight datasets.

Combining graph and SMILES modalities improves prediction accuracy.

Cross-attention effectively leverages complementary information from both representations.

Abstract

Molecular property prediction has attracted substantial attention recently. Accurate prediction of drug properties relies heavily on effective molecular representations. The structures of chemical compounds are commonly represented as graphs or SMILES sequences. Recent advances in learning drug properties commonly employ Graph Neural Networks (GNNs) based on the graph representation. For the SMILES representation, Transformer-based architectures have been adopted by treating each SMILES string as a sequence of tokens. Because each representation has its own advantages and disadvantages, combining both representations in learning drug properties is a promising direction. We propose a method named Dual-Modality Cross-Attention (DMCA) that can effectively combine the strengths of two representations by employing the cross-attention mechanism. DMCA was evaluated across eight datasets including both classification and regression tasks. Results show that our method achieves the best overall performance, highlighting its effectiveness in leveraging the complementary information from both graph and SMILES modalities.