Advancing Molecular Graph-Text Pre-training via Fine-grained Alignment

TL;DR

FineMolTex introduces a novel pre-training framework that jointly learns coarse and fine-grained molecular graph-text representations, significantly improving performance on molecular understanding and editing tasks.

Contribution

It proposes a new fine-grained molecular graph-text pre-training method that captures motif-level details alongside molecule-level knowledge, enhancing generalization.

Findings

Achieves up to 230% improvement in text-based molecule editing.

Effectively captures motif-level knowledge for molecular understanding.

Demonstrates potential for drug discovery and catalyst design.

Abstract

Understanding molecular structure and related knowledge is crucial for scientific research. Recent studies integrate molecular graphs with their textual descriptions to enhance molecular representation learning. However, they focus on the whole molecular graph and neglect frequently occurring subgraphs, known as motifs, which are essential for determining molecular properties. Without such fine-grained knowledge, these models struggle to generalize to unseen molecules and tasks that require motif-level insights. To bridge this gap, we propose FineMolTex, a novel Fine-grained Molecular graph-Text pre-training framework to jointly learn coarse-grained molecule-level knowledge and fine-grained motif-level knowledge. Specifically, FineMolTex consists of two pre-training tasks: a contrastive alignment task for coarse-grained matching and a masked multi-modal modeling task for fine-grained matching. In particular, the latter predicts the labels of masked motifs and words, which are selected based on their importance. By leveraging insights from both modalities, FineMolTex is able to understand the fine-grained matching between motifs and words. Finally, we conduct extensive experiments across three downstream tasks, achieving up to 230% improvement in the text-based molecule editing task. Additionally, our case studies reveal that FineMolTex successfully captures fine-grained knowledge, potentially offering valuable insights for drug discovery and catalyst design.