Espresso: High Compression For Rich Extraction From Videos for Your Vision-Language Model

TL;DR

Espresso is a novel video compression architecture that separately compresses spatial and temporal features into fixed-length sequences, enabling efficient and scalable video-language modeling without sacrificing reasoning capabilities.

Contribution

Introduces Espresso, a new architecture for fixed-length video feature compression that improves efficiency and reasoning in vision-language models.

Findings

Fixed-length compression enables scalable video processing.

Segment-wise processing maintains reasoning performance.

Competitive results against pooling-based approaches.

Abstract

Recent advances in vision-language models (VLMs) have shown great promise in connecting images and text, but extending these models to long videos remains challenging due to the rapid growth in token counts. Models that compress videos by local aggregation in time or space have become popular for handling long-form inputs; however, these pooling-based projectors sacrifice the benefits of fixed-length representations that are crucial for streaming and efficient video understanding. We introduce $\texttt{Espresso}$, a new architecture that separately compresses spatial and temporal features into fixed-length sequences. $\texttt{Espresso}$ enables efficient video encoding while maintaining strong long-form reasoning capabilities. Experiments show that fixed-length compression combined with segment-wise processing offers a scalable and competitive alternative to pooling-based approaches. Our results demonstrate that fixed-length projectors, when properly designed and trained, remain a viable foundation for video-language modeling.