Almost Linear Convergence under Minimal Score Assumptions: Quantized Transition Diffusion

TL;DR

This paper introduces Quantized Transition Diffusion (QTD), a novel diffusion approach that combines data quantization with discrete dynamics, enabling efficient, unbiased data generation with minimal score evaluations and theoretical guarantees.

Contribution

QTD unifies discrete and continuous diffusion models, supporting long-range transitions and providing provable unbiased sampling under minimal assumptions.

Findings

Achieves $O(d\,\ln^2(d/\epsilon))$ score evaluations for target distribution approximation.

Supports long-range data space transitions via Hamming distance-based Markov chain.

Provides theoretical guarantees for unbiased sampling with minimal score assumptions.

Abstract

Continuous diffusion models have demonstrated remarkable performance in data generation across various domains, yet their efficiency remains constrained by two critical limitations: (1) the local adjacency structure of the forward Markov process, which restricts long-range transitions in the data space, and (2) inherent biases introduced during the simulation of time-inhomogeneous reverse denoising processes. To address these challenges, we propose Quantized Transition Diffusion (QTD), a novel approach that integrates data quantization with discrete diffusion dynamics. Our method first transforms the continuous data distribution $p_*$ into a discrete one $q_*$ via histogram approximation and binary encoding, enabling efficient representation in a structured discrete latent space. We then design a continuous-time Markov chain (CTMC) with Hamming distance-based transitions as the forward process, which inherently supports long-range movements in the original data space. For reverse-time sampling, we introduce a \textit{truncated uniformization} technique to simulate the reverse CTMC, which can provably provide unbiased generation from $q_*$ under minimal score assumptions. Through a novel KL dynamic analysis of the reverse CTMC, we prove that QTD can generate samples with $O(d\ln^2(d/\epsilon))$ score evaluations in expectation to approximate the $d$--dimensional target distribution $p_*$ within an $\epsilon$ error tolerance. Our method not only establishes state-of-the-art inference efficiency but also advances the theoretical foundations of diffusion-based generative modeling by unifying discrete and continuous diffusion paradigms.