Generating stable molecules using imitation and reinforcement learning

TL;DR

This paper introduces a reinforcement learning method that generates stable molecules in 3D space, improving the discovery of low-energy, stable compounds by combining imitation learning and quantum chemical predictions.

Contribution

It presents a novel reinforcement learning framework that generates molecules in Cartesian coordinates, incorporating quantum chemical stability predictions, and enhances sample efficiency through imitation learning.

Findings

Successfully identifies low-energy molecules in the database.

Generates novel isomers not present in training data.

Refines molecule generation for larger, more complex molecules.

Abstract

Chemical space is routinely explored by machine learning methods to discover interesting molecules, before time-consuming experimental synthesizing is attempted. However, these methods often rely on a graph representation, ignoring 3D information necessary for determining the stability of the molecules. We propose a reinforcement learning approach for generating molecules in cartesian coordinates allowing for quantum chemical prediction of the stability. To improve sample-efficiency we learn basic chemical rules from imitation learning on the GDB-11 database to create an initial model applicable for all stoichiometries. We then deploy multiple copies of the model conditioned on a specific stoichiometry in a reinforcement learning setting. The models correctly identify low energy molecules in the database and produce novel isomers not found in the training set. Finally, we apply the model to larger molecules to show how reinforcement learning further refines the imitation learning model in domains far from the training data.