Grammars and reinforcement learning for molecule optimization

TL;DR

This paper introduces a simplified method for generating valid SMILES strings using a new grammar and applies reinforcement learning with a Transformer model to optimize molecular properties efficiently, outperforming previous methods.

Contribution

It presents a novel grammar-based approach for valid molecule generation and casts molecule optimization as a reinforcement learning problem with improved efficiency and performance.

Findings

Outperforms previous state-of-the-art baselines

Enables larger molecule generation with fewer model steps

Applicable to graph structure optimization with complex constraints

Abstract

We seek to automate the design of molecules based on specific chemical properties. Our primary contributions are a simpler method for generating SMILES strings guaranteed to be chemically valid, using a combination of a new context-free grammar for SMILES and additional masking logic; and casting the molecular property optimization as a reinforcement learning problem, specifically best-of-batch policy gradient applied to a Transformer model architecture. This approach uses substantially fewer model steps per atom than earlier approaches, thus enabling generation of larger molecules, and beats previous state-of-the art baselines by a significant margin. Applying reinforcement learning to a combination of a custom context-free grammar with additional masking to enforce non-local constraints is applicable to any optimization of a graph structure under a mixture of local and nonlocal constraints.