Em-Garde: A Propose-Match Framework for Proactive Streaming Video Understanding

TL;DR

Em-Garde introduces a decoupled, efficient framework for proactive streaming video understanding that improves response accuracy and computational efficiency by separating semantic understanding from perception and using structured proposals.

Contribution

The paper presents Em-Garde, a novel framework that decouples semantic understanding from streaming perception, enabling more efficient and accurate proactive video responses.

Findings

Outperforms prior models in accuracy on StreamingBench and OVO-Bench.

Achieves higher efficiency with reduced computational costs.

Validates effectiveness under strict computational constraints.

Abstract

Recent advances in Streaming Video Understanding has enabled a new interaction paradigm where models respond proactively to user queries. Current proactive VideoLLMs rely on per-frame triggering decision making, which suffers from an efficiency-accuracy dilemma. We propose Em-Garde, a novel framework that decouples semantic understanding from streaming perception. At query time, the Instruction-Guided Proposal Parser transforms user queries into structured, perceptually grounded visual proposals; during streaming, a Lightweight Proposal Matching Module performs efficient embedding-based matching to trigger responses. Experiments on StreamingBench and OVO-Bench demonstrate consistent improvements over prior models in proactive response accuracy and efficiency, validating an effective solution for proactive video understanding under strict computational constraints.