Decouple and Cache: KV Cache Construction for Streaming Video Understanding

TL;DR

This paper introduces DSCache, a training-free mechanism for constructing key-value caches in streaming video understanding, enabling models to process unbounded streams efficiently and accurately.

Contribution

The paper presents DSCache, a novel cache construction method that decouples past and current caches and incorporates position-agnostic encoding, improving streaming video model performance.

Findings

Achieves 2.5% accuracy improvement over prior methods on Streaming Video QA benchmarks.

Effectively maintains and updates caches for unbounded video streams.

Supports position extrapolation beyond training length, preventing position overflow.

Abstract

Streaming video understanding requires processing unbounded video streams with limited memory and computation, posing two key challenges. First, continuously constructing new and evicting old key-value(KV) caches is required for unbounded streams. Secondly, due to the high cost of collecting and training on unbounded streams, models must learn from short sequences while generalizing to long streams. Existing streaming VideoVLLMs fail to scale to unbounded video streams or focus on cache reuse strategies, leaving the impact of cache construction underexplored. In this paper, we propose Decoupled Streaming Cache(DSCache), a training-free cache construction mechanism that adapts pretrained offline models to streaming settings. DSCache maintains a cumulative past KV cache while constructing a separate instant cache on-demand, decoupled from past caches to preserve the informativeness of recent inputs. To enable position extrapolation beyond the training length, DSCache further incorporates a position-agnostic encoding strategy, ensuring KV caches to support unseen positions and preventing position overflow. Experiments on Streaming Video QA benchmarks demonstrate DSCache's state-of-the-art performance, with an average 2.5% accuracy gains over prior methods.