CROP: Integrating Topological and Spatial Structures via Cross-View Prefixes for Molecular LLMs

TL;DR

This paper introduces CROP, a method that integrates topological and spatial molecular views into large language models using efficient prefix resampling, significantly improving molecular understanding and prediction tasks.

Contribution

CROP is the first framework to effectively combine topological and spatial molecular views into LLMs via prefix resampling, enhancing multi-view integration and model performance.

Findings

CROP outperforms existing methods in molecule captioning.

CROP improves accuracy in IUPAC name prediction.

CROP enhances molecular property prediction results.

Abstract

Recent advances in molecular science have been propelled significantly by large language models (LLMs). However, their effectiveness is limited when relying solely on molecular sequences, which fail to capture the complex structures of molecules. Beyond sequence representation, molecules exhibit two complementary structural views: the first focuses on the topological relationships between atoms, as exemplified by the graph view; and the second emphasizes the spatial configuration of molecules, as represented by the image view. The two types of views provide unique insights into molecular structures. To leverage these views collaboratively, we propose the CROss-view Prefixes (CROP) to enhance LLMs' molecular understanding through efficient multi-view integration. CROP possesses two advantages: (i) efficiency: by jointly resampling multiple structural views into fixed-length prefixes, it avoids excessive consumption of the LLM's limited context length and allows easy expansion to more views; (ii) effectiveness: by utilizing the LLM's self-encoded molecular sequences to guide the resampling process, it boosts the quality of the generated prefixes. Specifically, our framework features a carefully designed SMILES Guided Resampler for view resampling, and a Structural Embedding Gate for converting the resulting embeddings into LLM's prefixes. Extensive experiments demonstrate the superiority of CROP in tasks including molecule captioning, IUPAC name prediction and molecule property prediction.