Search Tracker: Human-derived object tracking in-the-wild through large-scale search and retrieval

TL;DR

This paper introduces a novel search and retrieval approach leveraging human-annotated video libraries to improve object tracking in complex, real-world videos, outperforming existing appearance-based methods.

Contribution

It presents a new method that uses large-scale human-annotated video libraries and motion-based document retrieval for object tracking in unseen videos.

Findings

Outperforms state-of-the-art appearance-based trackers on in-the-wild datasets.

Introduces a new challenging dataset with complex appearance changes.

Validates the effectiveness of motion-based retrieval in real-world scenarios.

Abstract

Humans use context and scene knowledge to easily localize moving objects in conditions of complex illumination changes, scene clutter and occlusions. In this paper, we present a method to leverage human knowledge in the form of annotated video libraries in a novel search and retrieval based setting to track objects in unseen video sequences. For every video sequence, a document that represents motion information is generated. Documents of the unseen video are queried against the library at multiple scales to find videos with similar motion characteristics. This provides us with coarse localization of objects in the unseen video. We further adapt these retrieved object locations to the new video using an efficient warping scheme. The proposed method is validated on in-the-wild video surveillance datasets where we outperform state-of-the-art appearance-based trackers. We also introduce a new challenging dataset with complex object appearance changes.