TwinBooster: Synergising Large Language Models with Barlow Twins and Gradient Boosting for Enhanced Molecular Property Prediction

TL;DR

TwinBooster combines large language models, Barlow Twins, and gradient boosting to improve molecular property prediction, especially in data-scarce scenarios, enabling zero-shot learning and accelerating drug discovery.

Contribution

The paper introduces a novel architecture integrating LLMs with Barlow Twins and gradient boosting for enhanced molecular property prediction, including zero-shot capabilities.

Findings

State-of-the-art performance on FS-Mol benchmark

Effective zero-shot prediction for unseen bioassays and molecules

Demonstrates deep learning's potential in data-scarce property prediction

Abstract

The success of drug discovery and development relies on the precise prediction of molecular activities and properties. While in silico molecular property prediction has shown remarkable potential, its use has been limited so far to assays for which large amounts of data are available. In this study, we use a fine-tuned large language model to integrate biological assays based on their textual information, coupled with Barlow Twins, a Siamese neural network using a novel self-supervised learning approach. This architecture uses both assay information and molecular fingerprints to extract the true molecular information. TwinBooster enables the prediction of properties of unseen bioassays and molecules by providing state-of-the-art zero-shot learning tasks. Remarkably, our artificial intelligence pipeline shows excellent performance on the FS-Mol benchmark. This breakthrough demonstrates the application of deep learning to critical property prediction tasks where data is typically scarce. By accelerating the early identification of active molecules in drug discovery and development, this method has the potential to help streamline the identification of novel therapeutics.