Accelerating Multi-Condition T2I Generation via Adaptive Condition Offloading and Pruning

TL;DR

This paper introduces an end-edge system that accelerates multi-condition text-to-image generation by adaptively offloading conditions and pruning insignificant ones, reducing latency and improving quality.

Contribution

It proposes a novel Subtask Manager and Conditioning Scale Estimator to optimize condition processing, balancing local and edge computation for faster generation.

Findings

Latency reduced by nearly 25%

Average generation quality improved by 6%

Outperforms existing benchmarks in speed and quality

Abstract

Text-to-image (T2I) generation using multiple conditions enables fine-grained user control on the generated image. Yet, incorporating multi-condition inputs incurs substantial computation and communication overhead, due to additional preprocessing subtasks and control optimizations. It hence leads to unacceptable generation latency. In this paper, we propose an end-edge collaborative system design to accelerate multi-condition T2I generation through adaptive condition offloading and pruning. Extensive offline profiling reveal that, different conditions exhibit significant diversity in computation and communication costs. To this end, we propose a \textit{Subtask Manager} that jointly optimizes condition inference offloading and bandwidth allocation using a heuristic algorithm, balancing local and edge execution delays to minimize overall preprocessing latency. Then, we design a lightweight feature-driven \textit{Conditioning Scale Estimator} that evaluates the contribution of each condition by analyzing its feature activation strength and overlap with other conditions. This allows adaptive conditioning scale selection and pruning of insignificant conditions, thereby accelerating the denoising process. Extensive experimental results show that our system reduces latency by nearly 25\% and improves 6\% average generation quality, outperforming other benchmarks.