Entropy-Reinforced Planning with Large Language Models for Drug Discovery

TL;DR

This paper introduces ERP, an entropy-reinforced planning method for Transformer decoding in drug discovery, which improves molecule generation quality by balancing exploration and exploitation, outperforming existing methods across multiple benchmarks.

Contribution

ERP is a novel entropy-reinforced planning algorithm that enhances Transformer decoding for molecule generation, achieving consistent improvements over state-of-the-art methods.

Findings

ERP outperforms current state-of-the-art algorithms by 1-5% in drug discovery benchmarks.

ERP demonstrates robustness across different Transformer models and objectives.

ERP also surpasses existing methods in code generation benchmarks.

Abstract

The objective of drug discovery is to identify chemical compounds that possess specific pharmaceutical properties toward a binding target. Existing large language models (LLMS) can achieve high token matching scores in terms of likelihood for molecule generation. However, relying solely on LLM decoding often results in the generation of molecules that are either invalid due to a single misused token, or suboptimal due to unbalanced exploration and exploitation as a consequence of the LLMs prior experience. Here we propose ERP, Entropy-Reinforced Planning for Transformer Decoding, which employs an entropy-reinforced planning algorithm to enhance the Transformer decoding process and strike a balance between exploitation and exploration. ERP aims to achieve improvements in multiple properties compared to direct sampling from the Transformer. We evaluated ERP on the SARS-CoV-2 virus (3CLPro) and human cancer cell target protein (RTCB) benchmarks and demonstrated that, in both benchmarks, ERP consistently outperforms the current state-of-the-art algorithm by 1-5 percent, and baselines by 5-10 percent, respectively. Moreover, such improvement is robust across Transformer models trained with different objectives. Finally, to further illustrate the capabilities of ERP, we tested our algorithm on three code generation benchmarks and outperformed the current state-of-the-art approach as well. Our code is publicly available at: https://github.com/xuefeng-cs/ERP.