VQ-SAD: Vector Quantized Structure Aware Diffusion For Molecule Generation

TL;DR

VQ-SAD introduces a neuro-symbolic diffusion model for molecule generation that leverages vector quantized atom and bond codes as latent variables, improving over existing methods on standard datasets.

Contribution

It proposes a novel VQ-VAE based framework that uses codebooks as tokenizers for diffusion, combining symbolic and neural information for better molecule generation.

Findings

VQ-SAD slightly outperforms state-of-the-art models on QM9 and ZINC250k datasets.

The large discrete code space improves the denoising process in diffusion models.

Using frozen pretrained VQ-VAE enhances molecule generation quality.

Abstract

Many diffusion based molecule generation methods ignore the symbolic information of molecules and represent the atom and bond type as one hot representation. Methods based on Morgan fingerprints produce hash collisions and are hard to embed into a continuous space without information loss and random fingerprints correspond to no valid molecule. To circumvent this issue we use another paradigm and consider atom and bond codes as latent variables of VQ-VAE. We introduce VQ-SAD which first trains a VQ-VAE and uses the frozen pretrained VQ-VAE model and considers the codebooks for both atom and bond types as tokenizers for the downstream diffusion process. VQ-SAD is a neuro-symbolic model that utilizes both symbolic and neural structural information for a diffusion based model with learnable forward process. The large discrete code space provides a more balanced atom and bond types which enhances the denoising process. VQ-VAE slightly outperforms SOTA models for diffusion based molecule generation on QM9 and ZINC250k datasets.