Beyond Chemical Language: A Multimodal Approach to Enhance Molecular Property Prediction

TL;DR

This paper introduces MULTIMODAL-MOLFORMER, a multimodal model combining chemical language and physicochemical features with causal feature selection, significantly improving molecular property prediction accuracy over existing methods.

Contribution

The paper presents a novel multimodal approach with causal feature selection that enhances molecular property prediction by integrating chemical language embeddings with physicochemical features.

Findings

Outperforms state-of-the-art algorithms in biodegradability and toxicity prediction.

Effectively reduces dimensionality of physicochemical features while maintaining performance.

Demonstrates the benefit of combining chemical language and physicochemical data for molecular prediction.

Abstract

We present a novel multimodal language model approach for predicting molecular properties by combining chemical language representation with physicochemical features. Our approach, MULTIMODAL-MOLFORMER, utilizes a causal multistage feature selection method that identifies physicochemical features based on their direct causal effect on a specific target property. These causal features are then integrated with the vector space generated by molecular embeddings from MOLFORMER. In particular, we employ Mordred descriptors as physicochemical features and identify the Markov blanket of the target property, which theoretically contains the most relevant features for accurate prediction. Our results demonstrate a superior performance of our proposed approach compared to existing state-of-the-art algorithms, including the chemical language-based MOLFORMER and graph neural networks, in predicting complex tasks such as biodegradability and PFAS toxicity estimation. Moreover, we demonstrate the effectiveness of our feature selection method in reducing the dimensionality of the Mordred feature space while maintaining or improving the model's performance. Our approach opens up promising avenues for future research in molecular property prediction by harnessing the synergistic potential of both chemical language and physicochemical features, leading to enhanced performance and advancements in the field.