Strategies for Searching Video Content with Text Queries or Video Examples

TL;DR

This paper introduces novel strategies for content-based video retrieval that improve accuracy and speed when searching videos using text queries or video examples, addressing metadata scarcity issues.

Contribution

The paper presents new methods in feature design, fusion, and semantic detection that enhance CBVR performance for different query types, validated through TRECVID 2014 evaluations.

Findings

Outperformed other submissions in TRECVID 2014 in both text and video query searches.

Enhanced retrieval accuracy and speed with proposed strategies.

Demonstrated effectiveness of CBVR improvements in real-world evaluation.

Abstract

The large number of user-generated videos uploaded on to the Internet everyday has led to many commercial video search engines, which mainly rely on text metadata for search. However, metadata is often lacking for user-generated videos, thus these videos are unsearchable by current search engines. Therefore, content-based video retrieval (CBVR) tackles this metadata-scarcity problem by directly analyzing the visual and audio streams of each video. CBVR encompasses multiple research topics, including low-level feature design, feature fusion, semantic detector training and video search/reranking. We present novel strategies in these topics to enhance CBVR in both accuracy and speed under different query inputs, including pure textual queries and query by video examples. Our proposed strategies have been incorporated into our submission for the TRECVID 2014 Multimedia Event Detection evaluation, where our system outperformed other submissions in both text queries and video example queries, thus demonstrating the effectiveness of our proposed approaches.