Binomial flows: Denoising and flow matching for discrete ordinal data

TL;DR

This paper introduces Binomial flows, a novel framework for discrete diffusion modeling that enables denoising, sampling, and likelihood estimation for ordinal data, bridging a gap in discrete score-based generative modeling.

Contribution

The paper presents Binomial flows, the first discrete diffusion model that integrates denoising, sampling, and likelihood estimation for non-negative ordinal data.

Findings

Verified on synthetic data with successful denoising and sampling.

Achieved competitive results on real-world datasets.

Provided a simple training recipe for discrete diffusion models.

Abstract

Flow-based generative modeling in continuous spaces exploit Tweedie's formula to express the denoiser (learned in training) as a score function (used in sampling). In contrast, this relation has been largely missing in the discrete setting where common approaches focus on learning discrete scores and rates. In this work we close this gap for discrete non-negative ordinal data by introducing Binomial flows. Our framework provides a simple recipe for training a discrete diffusion model which simultaneously denoises, samples, and estimates exact likelihoods. We verify our methodology on synthetic examples and obtain competitive results on real-world data sets.