Quantum-inspired Reinforcement Learning for Synthesizable Drug Design

TL;DR

This paper introduces a quantum-inspired reinforcement learning approach with simulated annealing and genetic algorithms for synthesizable drug molecule design, outperforming existing methods in benchmark tests.

Contribution

It presents a novel quantum-inspired reinforcement learning method combining simulated annealing and genetic algorithms for molecular design.

Findings

Achieved competitive results on the PMO benchmark.

Outperformed state-of-the-art genetic algorithm methods.

Effectively navigated the chemical structure space.

Abstract

Synthesizable molecular design (also known as synthesizable molecular optimization) is a fundamental problem in drug discovery, and involves designing novel molecular structures to improve their properties according to drug-relevant oracle functions (i.e., objective) while ensuring synthetic feasibility. However, existing methods are mostly based on random search. To address this issue, in this paper, we introduce a novel approach using the reinforcement learning method with quantum-inspired simulated annealing policy neural network to navigate the vast discrete space of chemical structures intelligently. Specifically, we employ a deterministic REINFORCE algorithm using policy neural networks to output transitional probability to guide state transitions and local search using genetic algorithm to refine solutions to a local optimum within each iteration. Our methods are evaluated with the Practical Molecular Optimization (PMO) benchmark framework with a 10K query budget. We further showcase the competitive performance of our method by comparing it against the state-of-the-art genetic algorithms-based method.