Multi-Modal Molecular Representation Learning via Structure Awareness

TL;DR

This paper introduces MMSA, a novel multi-modal self-supervised framework that leverages structure-awareness and higher-order correlations to improve molecular representations for drug discovery, achieving state-of-the-art results.

Contribution

The paper proposes a structure-awareness-based multi-modal pre-training framework that models higher-order molecular relationships and invariant features, surpassing existing fusion approaches.

Findings

Achieves state-of-the-art ROC-AUC on MoleculeNet benchmark.

Improves performance by 1.8% to 9.6% over baseline methods.

Effectively captures higher-order and invariant molecular features.

Abstract

Accurate extraction of molecular representations is a critical step in the drug discovery process. In recent years, significant progress has been made in molecular representation learning methods, among which multi-modal molecular representation methods based on images, and 2D/3D topologies have become increasingly mainstream. However, existing these multi-modal approaches often directly fuse information from different modalities, overlooking the potential of intermodal interactions and failing to adequately capture the complex higher-order relationships and invariant features between molecules. To overcome these challenges, we propose a structure-awareness-based multi-modal self-supervised molecular representation pre-training framework (MMSA) designed to enhance molecular graph representations by leveraging invariant knowledge between molecules. The framework consists of two main modules: the multi-modal molecular representation learning module and the structure-awareness module. The multi-modal molecular representation learning module collaboratively processes information from different modalities of the same molecule to overcome intermodal differences and generate a unified molecular embedding. Subsequently, the structure-awareness module enhances the molecular representation by constructing a hypergraph structure to model higher-order correlations between molecules. This module also introduces a memory mechanism for storing typical molecular representations, aligning them with memory anchors in the memory bank to integrate invariant knowledge, thereby improving the model generalization ability. Extensive experiments have demonstrated the effectiveness of MMSA, which achieves state-of-the-art performance on the MoleculeNet benchmark, with average ROC-AUC improvements ranging from 1.8% to 9.6% over baseline methods.