Tracking the Untrackable

TL;DR

The paper introduces HAFT, a novel tracking model that predicts future visual features to improve robustness during occlusion, inspired by human anticipatory behavior and mental imagery.

Contribution

The paper proposes a hallucinating features approach that forecasts future frame embeddings, enhancing tracking performance under occlusion scenarios.

Findings

Improved tracking accuracy on multiple datasets.

Robustness against occlusion demonstrated.

Effective anticipation of target movement.

Abstract

Although short-term fully occlusion happens rare in visual object tracking, most trackers will fail under these circumstances. However, humans can still catch up the target by anticipating the trajectory of the target even the target is invisible. Recent psychology also has shown that humans build the mental image of the future. Inspired by that, we present a HAllucinating Features to Track (HAFT) model that enables to forecast the visual feature embedding of future frames. The anticipated future frames focus on the movement of the target while hallucinating the occluded part of the target. Jointly tracking on the hallucinated features and the real features improves the robustness of the tracker even when the target is highly occluded. Through extensive experimental evaluations, we achieve promising results on multiple datasets: OTB100, VOT2018, LaSOT, TrackingNet, and UAV123.