# VarDiU: A Variational Diffusive Upper Bound for One-Step Diffusion Distillation

## TL;DR

VarDiU introduces an unbiased gradient estimator for diffusion distillation, improving sample quality and training stability by directly optimizing a variational upper bound, surpassing previous methods like Diff-Instruct.

## Contribution

The paper proposes VarDiU, a novel variational diffusive upper bound that provides an unbiased gradient estimate for one-step diffusion distillation, enhancing performance and training efficiency.

## Key findings

- Achieves higher generation quality than previous methods.
- Enables more efficient and stable training process.
- Demonstrates effectiveness through comparison with Diff-Instruct.

## Abstract

Recently, diffusion distillation methods have compressed thousand-step teacher diffusion models into one-step student generators while preserving sample quality. Most existing approaches train the student model using a diffusive divergence whose gradient is approximated via the student's score function, learned through denoising score matching (DSM). Since DSM training is imperfect, the resulting gradient estimate is inevitably biased, leading to sub-optimal performance. In this paper, we propose VarDiU (pronounced /va:rdju:/), a Variational Diffusive Upper Bound that admits an unbiased gradient estimator and can be directly applied to diffusion distillation. Using this objective, we compare our method with Diff-Instruct and demonstrate that it achieves higher generation quality and enables a more efficient and stable training procedure for one-step diffusion distillation.