FIVR: Fine-grained Incident Video Retrieval

TL;DR

This paper defines the problem of Fine-grained Incident Video Retrieval (FIVR), introduces a large-scale dataset FIVR-200K, and evaluates state-of-the-art methods, highlighting challenges in retrieving related incident videos.

Contribution

It presents the FIVR problem, creates the FIVR-200K dataset with annotations for various association types, and benchmarks multiple retrieval methods.

Findings

State-of-the-art methods face significant challenges in FIVR tasks.

The FIVR-200K dataset enables comprehensive benchmarking.

Current methods show limited effectiveness on fine-grained incident retrieval.

Abstract

This paper introduces the problem of Fine-grained Incident Video Retrieval (FIVR). Given a query video, the objective is to retrieve all associated videos, considering several types of associations that range from duplicate videos to videos from the same incident. FIVR offers a single framework that contains several retrieval tasks as special cases. To address the benchmarking needs of all such tasks, we construct and present a large-scale annotated video dataset, which we call FIVR-200K, and it comprises 225,960 videos. To create the dataset, we devise a process for the collection of YouTube videos based on major news events from recent years crawled from Wikipedia and deploy a retrieval pipeline for the automatic selection of query videos based on their estimated suitability as benchmarks. We also devise a protocol for the annotation of the dataset with respect to the four types of video associations defined by FIVR. Finally, we report the results of an experimental study on the dataset comparing five state-of-the-art methods developed based on a variety of visual descriptors, highlighting the challenges of the current problem.