SFDDM: Single-fold Distillation for Diffusion models

TL;DR

SFDDM introduces a single-fold distillation method that efficiently compresses diffusion models into fewer steps, significantly reducing inference time while maintaining high-quality image synthesis.

Contribution

The paper proposes a novel single-fold distillation algorithm for diffusion models, enabling flexible and effective model compression in one step.

Findings

Student models can generate high-quality images with only 1% of original steps.

SFDDM outperforms multi-fold distillation in efficiency and quality.

The method preserves semantic consistency and enables meaningful image interpolation.

Abstract

While diffusion models effectively generate remarkable synthetic images, a key limitation is the inference inefficiency, requiring numerous sampling steps. To accelerate inference and maintain high-quality synthesis, teacher-student distillation is applied to compress the diffusion models in a progressive and binary manner by retraining, e.g., reducing the 1024-step model to a 128-step model in 3 folds. In this paper, we propose a single-fold distillation algorithm, SFDDM, which can flexibly compress the teacher diffusion model into a student model of any desired step, based on reparameterization of the intermediate inputs from the teacher model. To train the student diffusion, we minimize not only the output distance but also the distribution of the hidden variables between the teacher and student model. Extensive experiments on four datasets demonstrate that our student model trained by the proposed SFDDM is able to sample high-quality data with steps reduced to as little as approximately 1%, thus, trading off inference time. Our remarkable performance highlights that SFDDM effectively transfers knowledge in single-fold distillation, achieving semantic consistency and meaningful image interpolation.