IVCR-200K: A Large-Scale Multi-turn Dialogue Benchmark for Interactive Video Corpus Retrieval

TL;DR

This paper introduces IVCR-200K, a large-scale bilingual dataset and a multi-modal large language model framework for interactive, multi-turn video retrieval, addressing the lack of user-system interaction in previous methods.

Contribution

It presents the IVCR-200K dataset and a novel MLLM-based framework enabling interactive, multi-turn video retrieval with explainability, advancing personalized search capabilities.

Findings

IVCR-200K supports multi-turn, interactive video retrieval tasks.

The proposed framework improves retrieval accuracy and user interaction quality.

Experiments validate the effectiveness of the dataset and the framework.

Abstract

In recent years, significant developments have been made in both video retrieval and video moment retrieval tasks, which respectively retrieve complete videos or moments for a given text query. These advancements have greatly improved user satisfaction during the search process. However, previous work has failed to establish meaningful "interaction" between the retrieval system and the user, and its one-way retrieval paradigm can no longer fully meet the personalization and dynamic needs of at least 80.8\% of users. In this paper, we introduce the Interactive Video Corpus Retrieval (IVCR) task, a more realistic setting that enables multi-turn, conversational, and realistic interactions between the user and the retrieval system. To facilitate research on this challenging task, we introduce IVCR-200K, a high-quality, bilingual, multi-turn, conversational, and abstract semantic dataset that supports video retrieval and even moment retrieval. Furthermore, we propose a comprehensive framework based on multi-modal large language models (MLLMs) to help users interact in several modes with more explainable solutions. The extensive experiments demonstrate the effectiveness of our dataset and framework.