Grouped Speculative Decoding for Autoregressive Image Generation

TL;DR

Grouped Speculative Decoding (GSD) significantly speeds up autoregressive image models by evaluating clusters of tokens instead of single tokens, achieving 3.7x acceleration without extra training or quality loss.

Contribution

We introduce GSD, a training-free decoding acceleration method that leverages token clustering to improve inference speed of AR image models.

Findings

GSD achieves an average of 3.7x acceleration.

GSD maintains image quality comparable to baseline models.

Dynamic clustering outperforms static methods in token evaluation.

Abstract

Recently, autoregressive (AR) image models have demonstrated remarkable generative capabilities, positioning themselves as a compelling alternative to diffusion models. However, their sequential nature leads to long inference times, limiting their practical scalability. In this work, we introduce Grouped Speculative Decoding (GSD), a novel, training-free acceleration method for AR image models. While recent studies have explored Speculative Decoding (SD) as a means to speed up AR image generation, existing approaches either provide only modest acceleration or require additional training. Our in-depth analysis reveals a fundamental difference between language and image tokens: image tokens exhibit inherent redundancy and diversity, meaning multiple tokens can convey valid semantics. However, traditional SD methods are designed to accept only a single most-likely token, which fails to leverage this difference, leading to excessive false-negative rejections. To address this, we propose a new SD strategy that evaluates clusters of visually valid tokens rather than relying on a single target token. Additionally, we observe that static clustering based on embedding distance is ineffective, which motivates our dynamic GSD approach. Extensive experiments show that GSD accelerates AR image models by an average of 3.7x while preserving image quality-all without requiring any additional training. The source code is available at https://github.com/junhyukso/GSD