CoVR-R:Reason-Aware Composed Video Retrieval

TL;DR

This paper introduces CoVR-R, a reasoning-based, zero-shot approach for composed video retrieval that effectively captures implicit after-effects of edits, outperforming baselines and enhancing interpretability.

Contribution

It proposes a novel reasoning-first, zero-shot method leveraging large multimodal models for compositional video retrieval, along with a new benchmark for evaluating reasoning capabilities.

Findings

Outperforms strong retrieval baselines on recall at K.

Excels on implicit-effect subsets requiring reasoning.

Higher step consistency and effect factuality in retrieved videos.

Abstract

Composed Video Retrieval (CoVR) aims to find a target video given a reference video and a textual modification. Prior work assumes the modification text fully specifies the visual changes, overlooking after-effects and implicit consequences (e.g., motion, state transitions, viewpoint or duration cues) that emerge from the edit. We argue that successful CoVR requires reasoning about these after-effects. We introduce a reasoning-first, zero-shot approach that leverages large multimodal models to (i) infer causal and temporal consequences implied by the edit, and (ii) align the resulting reasoned queries to candidate videos without task-specific finetuning. To evaluate reasoning in CoVR, we also propose CoVR-Reason, a benchmark that pairs each (reference, edit, target) triplet with structured internal reasoning traces and challenging distractors that require predicting after-effects rather than keyword matching. Experiments show that our zero-shot method outperforms strong retrieval baselines on recall at K and particularly excels on implicit-effect subsets. Our automatic and human analysis confirm higher step consistency and effect factuality in our retrieved results. Our findings show that incorporating reasoning into general-purpose multimodal models enables effective CoVR by explicitly accounting for causal and temporal after-effects. This reduces dependence on task-specific supervision, improves generalization to challenging implicit-effect cases, and enhances interpretability of retrieval outcomes. These results point toward a scalable and principled framework for explainable video search. The model, code, and benchmark are available at https://github.com/mbzuai-oryx/CoVR-R.