SJD-VP: Speculative Jacobi Decoding with Verification Prediction for Autoregressive Image Generation

TL;DR

This paper introduces SJD-VP, a novel decoding method that improves the speed and quality of autoregressive image generation by predicting token acceptance based on probability changes across iterations.

Contribution

The paper proposes a new SJD-VP approach that leverages probability dynamics to enhance token acceptance and decoding efficiency in autoregressive image generation.

Findings

SJD-VP accelerates autoregressive decoding.

It improves image generation quality.

It can be integrated into existing SJD methods.

Abstract

Speculative Jacobi Decoding (SJD) has emerged as a promising method for accelerating autoregressive image generation. Despite its potential, existing SJD approaches often suffer from the low acceptance rate issue of speculative tokens due to token selection ambiguity. Recent works attempt to mitigate this issue primarily from the relaxed token verification perspective but fail to fully exploit the iterative dynamics of decoding. In this paper, we conduct an in-depth analysis and make a novel observation that tokens whose probabilities increase are more likely to match the verification-accepted and correct token. Based on this, we propose a novel Speculative Jacobi Decoding with Verification Prediction (SJD-VP). The key idea is to leverage the change in token probabilities across iterations to guide sampling, favoring tokens whose probabilities increase. This effectively predicts which tokens are likely to pass subsequent verification, boosting the acceptance rate. In particular, our SJD-VP is plug-and-play and can be seamlessly integrated into existing SJD methods. Extensive experiments on standard benchmarks demonstrate that our SJD-VP method consistently accelerates autoregressive decoding while improving image generation quality.