Multi-Objective Reinforcement Learning for Generating Covalent Inhibitor Candidates

TL;DR

This paper introduces a multi-objective reinforcement learning pipeline for generating covalent inhibitor candidates, successfully rediscovering known structures and exploring novel warhead motifs beyond training data.

Contribution

The study presents a novel RL-based generative approach that balances multiple properties and discovers new covalent warheads not present in training data.

Findings

Rediscovers known covalent inhibitors at rates up to 0.74%.

Generates structures with warhead-to-residue distances as short as 3.2 Å.

Spontaneously produces novel covalent warhead motifs supported by literature.

Abstract

Rational design of covalent inhibitors requires simultaneously optimizing multiple properties, such as binding affinity, target selectivity, or electrophilic reactivity. This presents a multi-objective problem not easily addressed by screening alone. Here we present a machine learning pipeline for generating covalent inhibitor candidates using multi-objective reinforcement learning (RL), applied to two targets: epidermal growth factor receptor (EGFR) and acetylcholinesterase (ACHE). A SMILES-based pretrained LSTM serves as the generative model, optimized via policy gradient RL with Pareto crowding distance to balance competing scoring functions including synthetic accessibility, predicted covalent activity, residue affinity, and an approximated docking score. The pipeline rediscovers known covalent inhibitors at rates of up to 0.50% (EGFR) and 0.74% (ACHE) in 10,000-structure runs, with candidate structures achieving warhead-to-residue distances as short as 5.5 angstrom (EGFR) and 3.2 angstrom (ACHE) after further docking-based screening. More notably, the pipeline spontaneously generates structures bearing warhead motifs absent from the training data - including allenes, 3-oxo-$\beta$-sultams, and $\alpha$-methylene-$\beta$-lactones - all of which have independent literature support as covalent warheads. These results suggest that RL-guided generation can explore covalent chemical space beyond its training distribution, and may be useful as a tool for medicinal chemists working on covalent drug discovery.