Repurformer: Transformers for Repurposing-Aware Molecule Generation

TL;DR

Repurformer is a novel transformer-based model that enhances molecule generation diversity by leveraging multi-hop protein-compound relationships and advanced pretraining techniques, addressing the sample bias problem in drug discovery.

Contribution

It introduces Repurformer, which combines bi-directional pretraining with FFT and LPF to improve diversity in molecule generation, a novel approach in this domain.

Findings

Successfully increases diversity of generated molecules

Creates effective substitutes resembling positive compounds

Addresses the sample bias problem in molecule generation

Abstract

Generating as diverse molecules as possible with desired properties is crucial for drug discovery research, which invokes many approaches based on deep generative models today. Despite recent advancements in these models, particularly in variational autoencoders (VAEs), generative adversarial networks (GANs), Transformers, and diffusion models, a significant challenge known as \textit{the sample bias problem} remains. This problem occurs when generated molecules targeting the same protein tend to be structurally similar, reducing the diversity of generation. To address this, we propose leveraging multi-hop relationships among proteins and compounds. Our model, Repurformer, integrates bi-directional pretraining with Fast Fourier Transform (FFT) and low-pass filtering (LPF) to capture complex interactions and generate diverse molecules. A series of experiments on BindingDB dataset confirm that Repurformer successfully creates substitutes for anchor compounds that resemble positive compounds, increasing diversity between the anchor and generated compounds.