Laminating Representation Autoencoders for Efficient Diffusion

TL;DR

This paper introduces FlatDINO, a variational autoencoder that compresses SSL patch features into a compact sequence, enabling efficient diffusion-based image generation with significantly reduced computational costs.

Contribution

FlatDINO is a novel autoencoder that reduces the dimensionality of SSL patch features, facilitating more efficient diffusion models for image synthesis.

Findings

FlatDINO compresses features by 8x in sequence length and 48x in total dimensionality.

Diffusion models on FlatDINO latents require 8x fewer FLOPs per forward pass.

Preliminary results show competitive image generation quality with reduced computational cost.

Abstract

Recent work has shown that diffusion models can generate high-quality images by operating directly on SSL patch features rather than pixel-space latents. However, the dense patch grids from encoders like DINOv2 contain significant redundancy, making diffusion needlessly expensive. We introduce FlatDINO, a variational autoencoder that compresses this representation into a one-dimensional sequence of just 32 continuous tokens -an 8x reduction in sequence length and 48x compression in total dimensionality. On ImageNet 256x256, a DiT-XL trained on FlatDINO latents achieves a gFID of 1.80 with classifier-free guidance while requiring 8x fewer FLOPs per forward pass and up to 4.5x fewer FLOPs per training step compared to diffusion on uncompressed DINOv2 features. These are preliminary results and this work is in progress.