Towards Cross-Modal Text-Molecule Retrieval with Better Modality Alignment

TL;DR

This paper introduces a novel cross-modal text-molecule retrieval model that enhances modality alignment by capturing second-order similarities and shared features, significantly improving retrieval accuracy in drug design applications.

Contribution

The paper proposes a new model with a memory bank feature projector and second-order similarity loss to better align text and molecule modalities, surpassing previous methods.

Findings

Achieves state-of-the-art performance, outperforming previous best by 6.4%.

Effectively captures second-order similarities for improved modality alignment.

Demonstrates strong experimental results validating the proposed approach.

Abstract

Cross-modal text-molecule retrieval model aims to learn a shared feature space of the text and molecule modalities for accurate similarity calculation, which facilitates the rapid screening of molecules with specific properties and activities in drug design. However, previous works have two main defects. First, they are inadequate in capturing modality-shared features considering the significant gap between text sequences and molecule graphs. Second, they mainly rely on contrastive learning and adversarial training for cross-modality alignment, both of which mainly focus on the first-order similarity, ignoring the second-order similarity that can capture more structural information in the embedding space. To address these issues, we propose a novel cross-modal text-molecule retrieval model with two-fold improvements. Specifically, on the top of two modality-specific encoders, we stack a memory bank based feature projector that contain learnable memory vectors to extract modality-shared features better. More importantly, during the model training, we calculate four kinds of similarity distributions (text-to-text, text-to-molecule, molecule-to-molecule, and molecule-to-text similarity distributions) for each instance, and then minimize the distance between these similarity distributions (namely second-order similarity losses) to enhance cross-modal alignment. Experimental results and analysis strongly demonstrate the effectiveness of our model. Particularly, our model achieves SOTA performance, outperforming the previously-reported best result by 6.4%.