Chemical Language Model Linker: blending text and molecules with modular adapters

TL;DR

ChemLML introduces a modular adapter-based approach to generate molecules from text by leveraging pretrained models, enabling efficient and flexible multi-modal molecule generation without training from scratch.

Contribution

It presents ChemLML, a lightweight adapter strategy that blends pretrained text and molecular models for conditional molecule generation, highlighting the impact of molecular representations.

Findings

SMILES representation often yields better generation performance.

ChemLML can generate candidate molecules for drug discovery tasks.

Filtered dataset improves evaluation reliability.

Abstract

The development of large language models and multi-modal models has enabled the appealing idea of generating novel molecules from text descriptions. Generative modeling would shift the paradigm from relying on large-scale chemical screening to find molecules with desired properties to directly generating those molecules. However, multi-modal models combining text and molecules are often trained from scratch, without leveraging existing high-quality pretrained models. Training from scratch consumes more computational resources and prohibits model scaling. In contrast, we propose a lightweight adapter-based strategy named Chemical Language Model Linker (ChemLML). ChemLML blends the two single domain models and obtains conditional molecular generation from text descriptions while still operating in the specialized embedding spaces of the molecular domain. ChemLML can tailor diverse pretrained text models for molecule generation by training relatively few adapter parameters. We find that the choice of molecular representation used within ChemLML, SMILES versus SELFIES, has a strong influence on conditional molecular generation performance. SMILES is often preferable despite not guaranteeing valid molecules. We raise issues in using the entire PubChem dataset of molecules and their associated descriptions for evaluating molecule generation and provide a filtered version of the dataset as a generation test set. To demonstrate how ChemLML could be used in practice, we generate candidate protein inhibitors and use docking to assess their quality and also generate candidate membrane permeable molecules.