Detecting Object Tracking Failure via Sequential Hypothesis Testing

TL;DR

This paper introduces a sequential hypothesis testing framework for real-time object tracking that provides formal safety assurances by reliably detecting tracking failures with minimal false alarms, applicable across various models and benchmarks.

Contribution

It proposes a novel, computationally efficient sequential testing method for tracking failure detection that is model-agnostic and does not require additional training.

Findings

Effective failure detection across multiple benchmarks

Provably controlled false alarm rates

Applicable to supervised and unsupervised tracking models

Abstract

Real-time online object tracking in videos constitutes a core task in computer vision, with wide-ranging applications including video surveillance, motion capture, and robotics. Deployed tracking systems usually lack formal safety assurances to convey when tracking is reliable and when it may fail, at best relying on heuristic measures of model confidence to raise alerts. To obtain such assurances we propose interpreting object tracking as a sequential hypothesis test, wherein evidence for or against tracking failures is gradually accumulated over time. Leveraging recent advancements in the field, our sequential test (formalized as an e-process) quickly identifies when tracking failures set in whilst provably containing false alerts at a desired rate, and thus limiting potentially costly re-calibration or intervention steps. The approach is computationally light-weight, requires no extra training or fine-tuning, and is in principle model-agnostic. We propose both supervised and unsupervised variants by leveraging either ground-truth or solely internal tracking information, and demonstrate its effectiveness for two established tracking models across four video benchmarks. As such, sequential testing can offer a statistically grounded and efficient mechanism to incorporate safety assurances into real-time tracking systems.