SJD-PAC: Accelerating Speculative Jacobi Decoding via Proactive Drafting and Adaptive Continuation

TL;DR

SJD-PAC enhances Speculative Jacobi Decoding for text-to-image synthesis by using proactive drafting and adaptive continuation, significantly increasing speed without quality loss.

Contribution

The paper introduces SJD-PAC, a novel framework that improves acceptance rates and inference speed in speculative decoding for image synthesis.

Findings

Achieves 3.8x speedup in inference

Maintains lossless image quality

Effective in high-entropy regions

Abstract

Speculative Jacobi Decoding (SJD) offers a draft-model-free approach to accelerate autoregressive text-to-image synthesis. However, the high-entropy nature of visual generation yields low draft-token acceptance rates in complex regions, creating a bottleneck that severely limits overall throughput. To overcome this, we introduce SJD-PAC, an enhanced SJD framework. First, SJD-PAC employs a proactive drafting strategy to improve local acceptance rates in these challenging high-entropy regions. Second, we introduce an adaptive continuation mechanism that sustains sequence validation after an initial rejection, bypassing the need for full resampling. Working in tandem, these optimizations significantly increase the average acceptance length per step, boosting inference speed while strictly preserving the target distribution. Experiments on standard text-to-image benchmarks demonstrate that SJD-PAC achieves a $3.8\times$ speedup with lossless image quality.