ComboStoc: Combinatorial Stochasticity for Diffusion Generative Models

TL;DR

This paper introduces ComboStoc, a method that leverages combinatorial stochastic processes to improve training speed and control in diffusion generative models across various data types.

Contribution

It proposes a novel stochastic process construction that fully exploits combinatorial structures, enhancing training efficiency and enabling flexible test-time generation.

Findings

Training speed significantly improved across images and 3D shapes.

Enables asynchronous test-time steps for better attribute control.

Addresses limitations of existing diffusion model training schemes.

Abstract

In this paper, we study an under-explored but important factor of diffusion generative models, i.e., the combinatorial complexity. Data samples are generally high-dimensional, and for various structured generation tasks, additional attributes are combined to associate with data samples. We show that the space spanned by the combination of dimensions and attributes can be insufficiently covered by existing training schemes of diffusion generative models, potentially limiting test time performance. We present a simple fix to this problem by constructing stochastic processes that fully exploit the combinatorial structures, hence the name ComboStoc. Using this simple strategy, we show that network training is significantly accelerated across diverse data modalities, including images and 3D structured shapes. Moreover, ComboStoc enables a new way of test time generation which uses asynchronous time steps for different dimensions and attributes, thus allowing for varying degrees of control over them. Our code is available at: https://github.com/Xrvitd/ComboStoc