Improving Training-free Conditional Diffusion Model via Fisher Information

TL;DR

This paper introduces a Fisher information-based approach to improve the speed and quality of training-free conditional diffusion models, reducing computational costs while maintaining high sample quality.

Contribution

It proposes a novel Fisher information-based method to reformulate the conditional term, enabling faster and more informative conditional diffusion sampling.

Findings

Up to 2x speed-up in generation time

Improved sample quality across various tasks

Reduced computational cost compared to baselines

Abstract

Training-free conditional diffusion models have received great attention in conditional image generation tasks. However, they require a computationally expensive conditional score estimator to let the intermediate results of each step in the reverse process toward the condition, which causes slow conditional generation. In this paper, we propose a novel Fisher information-based conditional diffusion (FICD) model to generate high-quality samples according to the condition. In particular, we further explore the conditional term from the perspective of Fisher information, where we show Fisher information can act as a weight to measure the informativeness of the condition in each generation step. According to this new perspective, we can control and gain more information along the conditional direction in the generation space. Thus, we propose the upper bound of the Fisher information to reformulate the conditional term, which increases the information gain and decreases the time cost. Experimental results also demonstrate that the proposed FICD can offer up to 2x speed-ups under the same sampling steps as most baselines. Meanwhile, FICD can improve the generation quality in various tasks compared to the baselines with a low computation cost.