Binary Diffusion Probabilistic Model

TL;DR

The paper introduces BDPM, a generative model tailored for binary data, which improves efficiency and performance in image tasks by using binary representations and XOR-based noise, outperforming existing methods in several benchmarks.

Contribution

BDPM is the first diffusion model designed specifically for binary data, utilizing XOR noise and binary embeddings to enhance image generation tasks.

Findings

Outperforms state-of-the-art methods on image super-resolution, inpainting, and restoration.

Achieves competitive results on ImageNet-1k with fewer parameters and sampling steps.

Reduces inference cost and accelerates convergence compared to traditional diffusion models.

Abstract

We propose the Binary Diffusion Probabilistic Model (BDPM), a generative framework specifically designed for data representations in binary form. Conventional denoising diffusion probabilistic models (DDPMs) assume continuous inputs, use mean squared error objectives and Gaussian perturbations, i.e., assumptions that are not suited to discrete and binary representations. BDPM instead encodes images into binary representations using multi bit-plane and learnable binary embeddings, perturbs them via XOR-based noise, and trains a model by optimizing a binary cross-entropy loss. These binary representations offer fine-grained noise control, accelerate convergence, and reduce inference cost. On image-to-image translation tasks, such as super-resolution, inpainting, and blind restoration, BDPM based on a small denoiser and multi bit-plane representation outperforms state-of-the-art methods on FFHQ, CelebA, and CelebA-HQ using a few sampling steps. In class-conditional generation on ImageNet-1k, BDPM based on learnable binary embeddings achieves competitive results among models with both low parameter counts and a few sampling steps.