From Prediction to Perfection: Introducing Refinement to Autoregressive Image Generation

TL;DR

This paper introduces TensorAR, a novel autoregressive image generation method that predicts overlapping image tensors to enable iterative refinement, significantly enhancing generation quality over traditional AR models.

Contribution

TensorAR reformulates AR image generation from token to tensor prediction, allowing iterative refinement and improving quality without retraining existing models.

Findings

TensorAR improves image generation quality across multiple datasets.

The discrete tensor noising scheme effectively prevents information leakage.

TensorAR is compatible with existing AR models as a plug-and-play module.

Abstract

Autoregressive (AR) image generators offer a language-model-friendly approach to image generation by predicting discrete image tokens in a causal sequence. However, unlike diffusion models, AR models lack a mechanism to refine previous predictions, limiting their generation quality. In this paper, we introduce TensorAR, a new AR paradigm that reformulates image generation from next-token prediction to next-tensor prediction. By generating overlapping windows of image patches (tensors) in a sliding fashion, TensorAR enables iterative refinement of previously generated content. To prevent information leakage during training, we propose a discrete tensor noising scheme, which perturbs input tokens via codebook-indexed noise. TensorAR is implemented as a plug-and-play module compatible with existing AR models. Extensive experiments on LlamaGEN, Open-MAGVIT2, and RAR demonstrate that TensorAR significantly improves the generation performance of autoregressive models.