Selecting Relevant Web Trained Concepts for Automated Event Retrieval

TL;DR

This paper introduces a novel event retrieval method that automatically discovers and prunes concept pairs based on relevance to improve performance in web video retrieval, addressing domain adaptation and calibration challenges.

Contribution

It proposes a new algorithm for selecting relevant concept pairs for event retrieval, enhancing accuracy by pruning irrelevant concepts and addressing domain adaptation issues.

Findings

Significant improvements over existing vision-based systems on TRECVID MED 13.

Effective pruning of irrelevant concepts enhances retrieval precision.

Addresses calibration and domain adaptation for unseen videos.

Abstract

Complex event retrieval is a challenging research problem, especially when no training videos are available. An alternative to collecting training videos is to train a large semantic concept bank a priori. Given a text description of an event, event retrieval is performed by selecting concepts linguistically related to the event description and fusing the concept responses on unseen videos. However, defining an exhaustive concept lexicon and pre-training it requires vast computational resources. Therefore, recent approaches automate concept discovery and training by leveraging large amounts of weakly annotated web data. Compact visually salient concepts are automatically obtained by the use of concept pairs or, more generally, n-grams. However, not all visually salient n-grams are necessarily useful for an event query--some combinations of concepts may be visually compact but irrelevant--and this drastically affects performance. We propose an event retrieval algorithm that constructs pairs of automatically discovered concepts and then prunes those concepts that are unlikely to be helpful for retrieval. Pruning depends both on the query and on the specific video instance being evaluated. Our approach also addresses calibration and domain adaptation issues that arise when applying concept detectors to unseen videos. We demonstrate large improvements over other vision based systems on the TRECVID MED 13 dataset.