Accelerating Image Generation with Sub-path Linear Approximation Model

TL;DR

This paper introduces SLAM, a novel method that accelerates diffusion-based image generation by approximating the diffusion process with sub-path linear ODEs, achieving high quality with fewer steps and faster inference.

Contribution

SLAM is the first to apply sub-path linear approximation to diffusion models, significantly reducing inference steps while maintaining high image quality.

Findings

SLAM achieves 2-4 step generation with high performance.

SLAM surpasses existing methods in FID scores.

Training requires only 6 A100 GPU days.

Abstract

Diffusion models have significantly advanced the state of the art in image, audio, and video generation tasks. However, their applications in practical scenarios are hindered by slow inference speed. Drawing inspiration from the approximation strategies utilized in consistency models, we propose the Sub-path Linear Approximation Model (SLAM), which accelerates diffusion models while maintaining high-quality image generation. SLAM treats the PF-ODE trajectory as a series of PF-ODE sub-paths divided by sampled points, and harnesses sub-path linear (SL) ODEs to form a progressive and continuous error estimation along each individual PF-ODE sub-path. The optimization on such SL-ODEs allows SLAM to construct denoising mappings with smaller cumulative approximated errors. An efficient distillation method is also developed to facilitate the incorporation of more advanced diffusion models, such as latent diffusion models. Our extensive experimental results demonstrate that SLAM achieves an efficient training regimen, requiring only 6 A100 GPU days to produce a high-quality generative model capable of 2 to 4-step generation with high performance. Comprehensive evaluations on LAION, MS COCO 2014, and MS COCO 2017 datasets also illustrate that SLAM surpasses existing acceleration methods in few-step generation tasks, achieving state-of-the-art performance both on FID and the quality of the generated images.