MolX: Enhancing Large Language Models for Molecular Understanding With A Multi-Modal Extension

TL;DR

MolX enhances large language models for molecular understanding by integrating multi-modal features from molecular graphs and fingerprints, significantly improving performance on molecule-related tasks with minimal additional training parameters.

Contribution

This work introduces MolX, a multi-modal extension that enables LLMs to better understand molecules by incorporating graph and fingerprint features, a novel approach in chemical domain modeling.

Findings

Outperforms baseline models on molecular tasks

Requires only 0.53% to 0.82% additional trainable parameters

Effective in both fine-tuned and zero-shot settings

Abstract

Large Language Models (LLMs) with their strong task-handling capabilities have shown remarkable advancements across a spectrum of fields, moving beyond natural language understanding. However, their proficiency within the chemistry domain remains restricted, especially in solving molecule-related tasks. This challenge is attributed to their inherent limitations in comprehending molecules using only common textual representations, i.e. SMILES strings. In this study, we seek to enhance the ability of LLMs to comprehend molecules by equipping them with a multi-modal external module, termed MolX. Instead of directly using SMILES strings to represent a molecule, we utilize specific encoders to extract fine-grained features from both SMILES string and 2D molecular graph representations for feeding into an LLM. A hand-crafted molecular fingerprint is incorporated to leverage its embedded domain knowledge. To establish an alignment between MolX and the LLM's textual input space, the model in which the LLM is frozen, is pre-trained with a strategy including a diverse set of tasks. Experimental evaluations show that our proposed method outperforms baselines across downstream molecule-related tasks ranging from molecule-to-text translation to molecular property prediction, with and without fine-tuning the LLM, while only introducing a small number of trainable parameters-0.53% and 0.82%, respectively.