MADTempo: An Interactive System for Multi-Event Temporal Video Retrieval with Query Augmentation

TL;DR

MADTempo is an innovative video retrieval system that combines temporal event modeling with web-scale visual grounding, enhancing accuracy and robustness in complex multi-event queries.

Contribution

It introduces a unified framework integrating temporal search with external visual data augmentation to improve multi-event video retrieval.

Findings

Enhanced temporal reasoning in video retrieval.

Improved handling of out-of-distribution queries.

Better retrieval accuracy on complex multi-event videos.

Abstract

The rapid expansion of video content across online platforms has accelerated the need for retrieval systems capable of understanding not only isolated visual moments but also the temporal structure of complex events. Existing approaches often fall short in modeling temporal dependencies across multiple events and in handling queries that reference unseen or rare visual concepts. To address these challenges, we introduce MADTempo, a video retrieval framework developed by our team, AIO_Trinh, that unifies temporal search with web-scale visual grounding. Our temporal search mechanism captures event-level continuity by aggregating similarity scores across sequential video segments, enabling coherent retrieval of multi-event queries. Complementarily, a Google Image Search-based fallback module expands query representations with external web imagery, effectively bridging gaps in pretrained visual embeddings and improving robustness against out-of-distribution (OOD) queries. Together, these components advance the temporal reasoning and generalization capabilities of modern video retrieval systems, paving the way for more semantically aware and adaptive retrieval across large-scale video corpora.