RLSynC: Offline-Online Reinforcement Learning for Synthon Completion

TL;DR

RLSynC is a novel offline-online reinforcement learning approach for synthon completion in retrosynthesis, combining offline training and online exploration to improve reaction prediction accuracy.

Contribution

It introduces a new offline-online RL framework for synthon completion, enhancing exploration and accuracy in semi-template-based retrosynthesis methods.

Findings

Outperforms state-of-the-art methods by up to 14.9%

Effectively explores new reaction spaces through combined offline and online learning

Uses a forward synthesis model to guide action search

Abstract

Retrosynthesis is the process of determining the set of reactant molecules that can react to form a desired product. Semi-template-based retrosynthesis methods, which imitate the reverse logic of synthesis reactions, first predict the reaction centers in the products, and then complete the resulting synthons back into reactants. We develop a new offline-online reinforcement learning method RLSynC for synthon completion in semi-template-based methods. RLSynC assigns one agent to each synthon, all of which complete the synthons by conducting actions step by step in a synchronized fashion. RLSynC learns the policy from both offline training episodes and online interactions, which allows RLSynC to explore new reaction spaces. RLSynC uses a standalone forward synthesis model to evaluate the likelihood of the predicted reactants in synthesizing a product, and thus guides the action search. Our results demonstrate that RLSynC can outperform state-of-the-art synthon completion methods with improvements as high as 14.9%, highlighting its potential in synthesis planning.