WeTok: Powerful Discrete Tokenization for High-Fidelity Visual Reconstruction

TL;DR

WeTok introduces a novel discrete visual tokenizer with group-wise quantization and a generative decoder, achieving high-fidelity image reconstruction at unprecedented compression ratios, outperforming previous methods on ImageNet.

Contribution

The paper presents WeTok, a new visual tokenizer with innovative group-wise lookup-free quantization and a generative decoder, significantly improving compression and reconstruction fidelity.

Findings

Achieves record-low zero-shot rFID of 0.12 on ImageNet at high fidelity.

Outperforms existing tokenizers like FLUX-VAE and SD-VAE in reconstruction quality.

Maintains high reconstruction fidelity at 768× compression ratio, surpassing prior methods.

Abstract

Visual tokenizer is a critical component for vision generation. However, the existing tokenizers often face unsatisfactory trade-off between compression ratios and reconstruction fidelity. To fill this gap, we introduce a powerful and concise WeTok tokenizer, which surpasses the previous leading tokenizers via two core innovations. (1) Group-wise lookup-free Quantization (GQ). We partition the latent features into groups, and perform lookup-free quantization for each group. As a result, GQ can efficiently overcome memory and computation limitations of prior tokenizers, while achieving a reconstruction breakthrough with more scalable codebooks. (2) Generative Decoder (GD). Different from prior tokenizers, we introduce a generative decoder with a prior of extra noise variable. In this case, GD can probabilistically model the distribution of visual data conditioned on discrete tokens, allowing WeTok to reconstruct visual details, especially at high compression ratio. On the ImageNet 50k validation set, at a high-fidelity setting, WeTok achieves a record-low zero-shot rFID of 0.12, outperforming leading continuous tokenizers like FLUX-VAE (0.18) and SD-VAE 3.5 (0.19) with 400% compression ratio. Furthermore, in a high-compression regime, WeTok achieves a zero-shot rFID of 3.49 at a 768$\times$ compression ratio, substantially surpassing Cosmos, which scores 4.57 at only 50% our compression ratio. Code and models are available: https://github.com/zhuangshaobin/WeTok.