End-to-End Autoregressive Image Generation with 1D Semantic Tokenizer

TL;DR

This paper introduces an end-to-end autoregressive image generation method that jointly trains a visual tokenizer and a generative model, achieving state-of-the-art results on ImageNet 256x256.

Contribution

It proposes a unified training pipeline for image tokenization and generation, improving over prior two-stage methods and leveraging vision foundation models.

Findings

Achieved a state-of-the-art FID score of 1.48 on ImageNet 256x256.

Demonstrated the effectiveness of end-to-end training for autoregressive image modeling.

Improved image generation quality without guidance.

Abstract

Autoregressive image modeling relies on visual tokenizers to compress images into compact latent representations. We design an end-to-end training pipeline that jointly optimizes reconstruction and generation, enabling direct supervision from generation results to the tokenizer. This contrasts with prior two-stage approaches that train tokenizers and generative models separately. We further investigate leveraging vision foundation models to improve 1D tokenizers for autoregressive modeling. Our autoregressive generative model achieves strong empirical results, including a state-of-the-art FID score of 1.48 without guidance on ImageNet 256x256 generation.