UniCompress: Token Compression for Unified Vision-Language Understanding and Generation

TL;DR

UniCompress is a lightweight, modular token compression method that reduces visual tokens in unified vision-language models, significantly improving efficiency with minimal performance loss.

Contribution

It introduces a novel plug-in compression mechanism guided by learnable meta tokens, enabling efficient token reduction without full model retraining.

Findings

Reduces visual tokens by up to 4 times

Improves inference latency and training cost

Maintains performance with minimal degradation

Abstract

Unified models aim to support both understanding and generation by encoding images into discrete tokens and processing them alongside text within a single autoregressive framework. This unified design offers architectural simplicity and cross-modal synergy, which facilitates shared parameterization, consistent training objectives, and seamless transfer between modalities. However, the large number of visual tokens required by such models introduces substantial computation and memory overhead, and this inefficiency directly hinders deployment in resource constrained scenarios such as embodied AI systems. In this work, we propose a unified token compression algorithm UniCompress that significantly reduces visual token count while preserving performance on both image understanding and generation tasks. Our method introduces a plug-in compression and decompression mechanism guided with learnable global meta tokens. The framework is lightweight and modular, enabling efficient integration into existing models without full retraining. Experimental results show that our approach reduces image tokens by up to 4 times, achieves substantial gains in inference latency and training cost, and incurs only minimal performance degradation, which demonstrates the promise of token-efficient unified modeling for real world multimodal applications.