Memory Based Online Learning of Deep Representations from Video Streams

TL;DR

This paper introduces an online unsupervised method for face identity learning from video streams that leverages deep descriptors, memory mechanisms, and feature management to improve face tracking and identification without future data.

Contribution

It proposes a novel memory-based online learning approach using deep face descriptors, a discriminative matching method, and a feature forgetting strategy for stable, real-time face recognition.

Findings

Achieves comparable face tracking results to offline methods.

Outperforms in face identification accuracy.

Demonstrates stability and effectiveness in unconstrained videos.

Abstract

We present a novel online unsupervised method for face identity learning from video streams. The method exploits deep face descriptors together with a memory based learning mechanism that takes advantage of the temporal coherence of visual data. Specifically, we introduce a discriminative feature matching solution based on Reverse Nearest Neighbour and a feature forgetting strategy that detect redundant features and discard them appropriately while time progresses. It is shown that the proposed learning procedure is asymptotically stable and can be effectively used in relevant applications like multiple face identification and tracking from unconstrained video streams. Experimental results show that the proposed method achieves comparable results in the task of multiple face tracking and better performance in face identification with offline approaches exploiting future information. Code will be publicly available.