Elastic Diffusion Transformer
Jiangshan Wang, Zeqiang Lai, Jiarui Chen, Jiayi Guo, Hang Guo, Xiu Li, Xiangyu Yue, Chunchao Guo

TL;DR
The paper introduces Elastic Diffusion Transformer (E-DiT), an adaptive framework that dynamically skips computations in diffusion models, significantly improving efficiency with minimal quality loss across image and 3D asset generation.
Contribution
E-DiT is the first adaptive acceleration method for DiT that uses sample-dependent sparsity and lightweight routers to skip blocks and reduce computation during inference.
Findings
Achieves up to 2x speedup in diffusion model generation.
Maintains high generation quality with negligible loss.
Effective across 2D images and 3D assets.
Abstract
Diffusion Transformers (DiT) have demonstrated remarkable generative capabilities but remain highly computationally expensive. Previous acceleration methods, such as pruning and distillation, typically rely on a fixed computational capacity, leading to insufficient acceleration and degraded generation quality. To address this limitation, we propose \textbf{Elastic Diffusion Transformer (E-DiT)}, an adaptive acceleration framework for DiT that effectively improves efficiency while maintaining generation quality. Specifically, we observe that the generative process of DiT exhibits substantial sparsity (i.e., some computations can be skipped with minimal impact on quality), and this sparsity varies significantly across samples. Motivated by this observation, E-DiT equips each DiT block with a lightweight router that dynamically identifies sample-dependent sparsity from the input latent.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Generative Adversarial Networks and Image Synthesis · Network Packet Processing and Optimization
