Integrated Semantic and Temporal Alignment for Interactive Video Retrieval
Thanh-Danh Luu, Le-Vu Nguyen Dinh, Duc-Thien Tran, Duy-Bao Bui, Nam-Tien Le, Tinh-Anh Nguyen Nhu

TL;DR
This paper presents an integrated video retrieval system combining semantic and temporal alignment techniques, featuring novel components for handling out-of-knowledge queries and temporal event alignment, advancing real-world video search capabilities.
Contribution
The paper introduces QUEST and DANTE, innovative modules for out-of-knowledge query resolution and temporal event alignment, integrated into a scalable video retrieval framework.
Findings
System effectively handles out-of-knowledge queries.
DANTE achieves efficient temporal alignment in complex videos.
Framework outperforms existing methods on TRAKE challenge.
Abstract
The growing volume of video data and the introduction of complex retrieval challenges, such as the Temporal Retrieval and Alignment of Key Events (TRAKE) task at the Ho Chi Minh City AI Challenge 2025, expose critical limitations in existing systems. Many methodologies lack scalable, holistic architectures and rely on "frozen" embedding models that fail on out-of-knowledge (OOK) or real-world queries. This paper introduces the comprehensive video retrieval framework developed by team AIO\_Owlgorithms to address these gaps. Our system features an architecture integrating TransNetV2 for scene segmentation, BEiT-3 for visual embeddings in Milvus, and Gemini OCR for metadata in Elasticsearch. We propose two components: (1) \textbf{QUEST} (Query Understanding and External Search for Out-of-Knowledge Tasks), a two-branch framework that leverages a Large Language Model (LLM) for query…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Video Analysis and Summarization · Advanced Image and Video Retrieval Techniques
