Interpolating Discrete Diffusion Models with Controllable Resampling

TL;DR

The paper introduces IDDM, a novel discrete diffusion model with controllable resampling that reduces error accumulation and improves generation quality across text and graph domains.

Contribution

IDDM's interpolating transitions and controllable resampling mechanism enhance discrete diffusion models by mitigating errors and decoupling training from inference.

Findings

IDDM achieves competitive results on molecular graph and text generation tasks.

The model effectively reduces error propagation compared to previous approaches.

Benchmark results show improved sample quality and diversity.

Abstract

Discrete diffusion models form a powerful class of generative models across diverse domains, including text and graphs. However, existing approaches face fundamental limitations. Masked diffusion models suffer from irreversible errors due to early unmasking, while uniform diffusion models, despite enabling self-correction, often yield low-quality samples due to their strong reliance on intermediate latent states. We introduce IDDM, an Interpolating Discrete Diffusion Model, that improves diffusion by reducing dependence on intermediate latent states. Central to IDDM is a controllable resampling mechanism that partially resets probability mass to the marginal distribution, mitigating error accumulation and enabling more effective token corrections. IDDM specifies a generative process whose transitions interpolate between staying at the current state, resampling from a prior, and flipping toward the target state, while enforcing marginal consistency and fully decoupling training from inference. We benchmark our model against state-of-the-art discrete diffusion models across molecular graph generation as well as text generation tasks, demonstrating competitive performance.