MaskBit: Embedding-free Image Generation via Bit Tokens

TL;DR

This paper introduces MaskBit, an embedding-free image generation method using bit tokens, achieving state-of-the-art results on ImageNet with a simplified model and providing a modernized VQGAN for improved image synthesis.

Contribution

The paper presents a systematic modernization of VQGANs and introduces a novel embedding-free generation network operating directly on bit tokens, advancing image synthesis techniques.

Findings

Achieved a new state-of-the-art FID of 1.52 on ImageNet 256x256.

Developed a compact generator with only 305 million parameters.

Provided a transparent and reproducible VQGAN model.

Abstract

Masked transformer models for class-conditional image generation have become a compelling alternative to diffusion models. Typically comprising two stages - an initial VQGAN model for transitioning between latent space and image space, and a subsequent Transformer model for image generation within latent space - these frameworks offer promising avenues for image synthesis. In this study, we present two primary contributions: Firstly, an empirical and systematic examination of VQGANs, leading to a modernized VQGAN. Secondly, a novel embedding-free generation network operating directly on bit tokens - a binary quantized representation of tokens with rich semantics. The first contribution furnishes a transparent, reproducible, and high-performing VQGAN model, enhancing accessibility and matching the performance of current state-of-the-art methods while revealing previously undisclosed details. The second contribution demonstrates that embedding-free image generation using bit tokens achieves a new state-of-the-art FID of 1.52 on the ImageNet 256x256 benchmark, with a compact generator model of mere 305M parameters. The code for this project is available on https://github.com/markweberdev/maskbit.