Simple Baselines for Interactive Video Retrieval with Questions and Answers

TL;DR

This paper introduces simple, effective baselines for interactive video retrieval using question-answering, demonstrating significant performance improvements on multiple datasets without complex models.

Contribution

Proposes straightforward baselines employing VideoQA models for interactive video retrieval, enabling productive study without ground truth dialogue data.

Findings

Question-based interaction improves retrieval performance

Framework tested on MSR-VTT, MSVD, AVSD datasets

Significant gains over non-interactive methods

Abstract

To date, the majority of video retrieval systems have been optimized for a "single-shot" scenario in which the user submits a query in isolation, ignoring previous interactions with the system. Recently, there has been renewed interest in interactive systems to enhance retrieval, but existing approaches are complex and deliver limited gains in performance. In this work, we revisit this topic and propose several simple yet effective baselines for interactive video retrieval via question-answering. We employ a VideoQA model to simulate user interactions and show that this enables the productive study of the interactive retrieval task without access to ground truth dialogue data. Experiments on MSR-VTT, MSVD, and AVSD show that our framework using question-based interaction significantly improves the performance of text-based video retrieval systems.