Materials Discovery with Extreme Properties via Reinforcement Learning-Guided Combinatorial Chemistry

TL;DR

This paper introduces a reinforcement learning-guided combinatorial chemistry approach for discovering new materials with extreme properties, outperforming traditional probability-based models in generating valid, high-quality molecules for practical applications.

Contribution

The authors develop a rule-based molecular design method guided by reinforcement learning, enabling the discovery of molecules with superior properties beyond existing models.

Findings

Successfully discovered 1,315 target-hitting molecules out of 100,000 trials.

Generated 100% chemically valid molecules under fragment binding rules.

Demonstrated effectiveness in discovering protein docking molecules and HIV inhibitors.

Abstract

The goal of most materials discovery is to discover materials that are superior to those currently known. Fundamentally, this is close to extrapolation, which is a weak point for most machine learning models that learn the probability distribution of data. Herein, we develop reinforcement learning-guided combinatorial chemistry, which is a rule-based molecular designer driven by trained policy for selecting subsequent molecular fragments to get a target molecule. Since our model has the potential to generate all possible molecular structures that can be obtained from combinations of molecular fragments, unknown molecules with superior properties can be discovered. We theoretically and empirically demonstrate that our model is more suitable for discovering better compounds than probability distribution-learning models. In an experiment aimed at discovering molecules that hit seven extreme target properties, our model discovered 1,315 of all target-hitting molecules and 7,629 of five target-hitting molecules out of 100,000 trials, whereas the probability distribution-learning models failed. Moreover, it has been confirmed that every molecule generated under the binding rules of molecular fragments is 100% chemically valid. To illustrate the performance in actual problems, we also demonstrate that our models work well on two practical applications: discovering protein docking molecules and HIV inhibitors.