XQ-GAN: An Open-source Image Tokenization Framework for Autoregressive Generation

TL;DR

XQ-GAN introduces a versatile image tokenization framework utilizing advanced quantization methods, significantly improving image reconstruction and generation quality on ImageNet, and providing pre-trained models for community use.

Contribution

The paper presents XQ-GAN, a flexible framework integrating multiple state-of-the-art quantization techniques for enhanced image tokenization in generative models.

Findings

Achieves an rFID of 0.64 on ImageNet 256x256, surpassing previous models.

Improves gFID metrics when used as a tokenizer, e.g., gFID of 2.6 with VAR.

Provides pre-trained tokenizers for community research and development.

Abstract

Image tokenizers play a critical role in shaping the performance of subsequent generative models. Since the introduction of VQ-GAN, discrete image tokenization has undergone remarkable advancements. Improvements in architecture, quantization techniques, and training recipes have significantly enhanced both image reconstruction and the downstream generation quality. In this paper, we present XQ-GAN, an image tokenization framework designed for both image reconstruction and generation tasks. Our framework integrates state-of-the-art quantization techniques, including vector quantization (VQ), residual quantization (RQ), multi-scale residual quantization (MSVQ), product quantization (PQ), lookup-free quantization (LFQ), and binary spherical quantization (BSQ), within a highly flexible and customizable training environment. On the standard ImageNet 256x256 benchmark, our released model achieves an rFID of 0.64, significantly surpassing MAGVIT-v2 (0.9 rFID) and VAR (0.9 rFID). Furthermore, we demonstrate that using XQ-GAN as a tokenizer improves gFID metrics alongside rFID. For instance, with the same VAR architecture, XQ-GAN+VAR achieves a gFID of 2.6, outperforming VAR's 3.3 gFID by a notable margin. To support further research, we provide pre-trained weights of different image tokenizers for the community to directly train the subsequent generative models on it or fine-tune for specialized tasks.