Interactive Multi-Turn Retrieval for Health Videos

TL;DR

This paper introduces an interactive multi-turn retrieval system for health videos, improving retrieval accuracy by incorporating follow-up query refinements and establishing a new benchmark dataset.

Contribution

It proposes DATR, a two-stage retrieval framework that enhances health video search through multi-turn query fusion and efficient coarse-to-fine retrieval, along with a new dataset MHVRC.

Findings

DATR outperforms strong baselines in retrieval accuracy.

User studies show multi-turn queries better capture procedural semantics.

MHVRC provides a new benchmark for health video retrieval.

Abstract

The growing availability of health-related instructional videos creates new opportunities for clinical training, patient rehabilitation, and health education, yet existing retrieval systems remain largely single-turn: a user submits one query and receives one ranked list. This interaction is brittle in health scenarios, where information needs are often vague at first and become clinically meaningful only after follow-up constraints such as posture, hand placement, contraindications, equipment, or patient condition are specified. We introduce interactive multi-turn semantic retrieval for health videos and construct MHVRC, a Multi-Turn Health Video Retrieval Corpus, by combining video-grounded descriptions from VideoChat-Flash with query refinements generated by DeepSeek. We further propose DATR, a Dialogue-Aware Two-Stage Retrieval framework. DATR first performs efficient coarse retrieval with a CLIP-style dual encoder and sparse frame sampling, then re-ranks the top candidates through multi-turn query fusion and a lightweight cross-encoder scoring module. Experiments on MHVRC show consistent gains over strong text-video retrieval baselines, while user studies indicate that refined multi-turn queries better capture fine-grained procedural semantics than single-turn annotations. The work establishes a benchmark and a scalable technical recipe for interactive health video retrieval.