Long-Term Face Tracking for Crowded Video-Surveillance Scenarios

TL;DR

This paper introduces a real-time long-term face tracking system designed for crowded, unconstrained video-surveillance scenarios, leveraging face detection and recognition to improve tracking duration and robustness.

Contribution

The paper proposes a novel long-term multi-face tracking architecture combining fast visual tracking with face verification for tracklet reconnection, suitable for crowded environments.

Findings

Achieves up to 50% longer tracks than existing deep learning trackers.

Introduces specialized metrics for evaluating long-term tracking.

Provides a new publicly available video dataset for long-term face tracking.

Abstract

Most current multi-object trackers focus on short-term tracking, and are based on deep and complex systems that do not operate in real-time, often making them impractical for video-surveillance. In this paper, we present a long-term multi-face tracking architecture conceived for working in crowded contexts, particularly unconstrained in terms of movement and occlusions, and where the face is often the only visible part of the person. Our system benefits from advances in the fields of face detection and face recognition to achieve long-term tracking. It follows a tracking-by-detection approach, combining a fast short-term visual tracker with a novel online tracklet reconnection strategy grounded on face verification. Additionally, a correction module is included to correct past track assignments with no extra computational cost. We present a series of experiments introducing novel, specialized metrics for the evaluation of long-term tracking capabilities and a video dataset that we publicly release. Findings demonstrate that, in this context, our approach allows to obtain up to 50% longer tracks than state-of-the-art deep learning trackers.