DINO-Tok: Adapting DINO for Visual Tokenizers

TL;DR

DINO-Tok introduces a novel visual tokenizer based on a frozen DINO encoder, unifying hierarchical features for high-fidelity, semantically consistent image generation, addressing challenges in high-dimensional latent spaces.

Contribution

It presents DINO-Tok, a new visual tokenizer that combines continuous autoencoding and vector quantization using a frozen DINO encoder, and proposes Dominant-Subspace Quantization to improve codebook stability.

Findings

Achieves 0.28 rFID in autoencoding on ImageNet 256x256

Attains 1.10 rFID with VQ for high-fidelity reconstruction

Demonstrates strong few-step generation performance with 1.82 gFID

Abstract

Recent advances in visual generation have emphasized the importance of Latent Generative Models (LGMs), which critically depend on effective visual tokenizers to bridge pixels and semantic representations. However, tokenizers constructed on pre-trained vision foundation models (VFMs) often struggle to balance semantic richness and reconstruction fidelity in high-dimensional latent spaces. In this paper, we introduce DINO-Tok, a visual tokenizer built upon a frozen DINO encoder that supports both continuous autoencoding (DINO-Tok-AE) and discrete vector-quantization (DINO-Tok-VQ). By unifying hierarchical representations from both shallow fine-grained features and deep global semantics into an information-complete latent space, DINO-Tok preserves texture details while maintaining \textit{semantic consistency} for generation. We further investigate VQ in frozen semantic feature spaces of high dimensionality, where information dilution and codebook collapse frequently arise. To address this issue, we propose Dominant-Subspace Quantization (DSQ), which leverages a global PCA analysis to select principal components while suppressing noisy dimensions, thereby stabilizing codebook optimization and improving reconstruction and generation quality. On ImageNet 256x256, DINO-Tok achieves strong reconstruction performance, achieving 0.28 rFID for continuous autoencoding and 1.10 rFID for discrete VQ, as well as strong few-step generation performance 1.82 gFID for diffusion and 2.44 gFID for autoregressive generation. These results demonstrate that pre-trained VFMs such as DINO can be directly adapted into high-fidelity, semantically aligned visual tokenizers for next-generation latent generative models. Code will be publicly available at https://github.com/MKJia/DINO-Tok.