Fine-tuning Pocket-Aware Diffusion Models via Denoising Policy Optimization

TL;DR

DEPPA introduces a reinforcement learning-based method to fine-tune pocket-aware diffusion models for multi-property molecule optimization in drug discovery.

Contribution

It presents DEPPA, a novel approach combining diffusion models and reinforcement learning for multi-property molecule optimization.

Findings

DEPPA outperforms baselines in binding affinity, drug-likeness, and diversity.

It achieves competitive synthesizability on the CrossDocked2020 benchmark.

The method enables fine-grained control over multiple molecular properties.

Abstract

Structure-based drug design has been accelerated by pocket-aware 3D generative models, yet most methods primarily fit the training distribution and may fall short of satisfying multiple properties required in real-world therapeutic drug discovery. Recently, increasing attention has focused on structure-based molecule optimization (SBMO), which targets fine-grained control over multiple specified molecular properties. In this paper, we present DEPPA, a novel SBMO approach building upon Denoising Diffusion Policy Optimization for fine-tuning a pre-trained pocket-aware diffusion model via reinforcement learning. DEPPA enables optimization over multiple properties, including binding affinity, drug-likeness, synthesizability and diversity. We formulate the reverse denoising process of the pretrained pocket-aware diffusion model as a multi-step Markov Decision Process, where the desired properties that serve as reward signals are evaluated on the final generated ligand molecules. DEPPA incorporates a coarse denoising scheduler during the RL fine-tuning to achieve efficient and effective molecule optimization. Experimental results on the CrossDocked2020 benchmark demonstrate that DEPPA outperforms baselines in binding affinity (Vina Score -8.5 kcal/mol), drug-likeness and diversity while exhibiting competitive performance in synthesizability. The source code is available at https://github.com/xy9485/DePPA .