Prototype Memory for Large-scale Face Representation Learning

TL;DR

This paper introduces Prototype Memory, a scalable and efficient method for large-scale face recognition that maintains up-to-date class prototypes in memory, improving training stability and accuracy on datasets with millions of identities.

Contribution

The paper proposes Prototype Memory, a novel memory module that dynamically updates class prototypes to address obsolescence in large-scale face recognition training.

Findings

Prototype Memory improves face recognition accuracy on benchmark datasets.

The method is computationally efficient and scalable to datasets with millions of identities.

Prototype Memory outperforms existing sampled softmax approaches in maintaining relevant class prototypes.

Abstract

Face representation learning using datasets with a massive number of identities requires appropriate training methods. Softmax-based approach, currently the state-of-the-art in face recognition, in its usual "full softmax" form is not suitable for datasets with millions of persons. Several methods, based on the "sampled softmax" approach, were proposed to remove this limitation. These methods, however, have a set of disadvantages. One of them is a problem of "prototype obsolescence": classifier weights (prototypes) of the rarely sampled classes receive too scarce gradients and become outdated and detached from the current encoder state, resulting in incorrect training signals. This problem is especially serious in ultra-large-scale datasets. In this paper, we propose a novel face representation learning model called Prototype Memory, which alleviates this problem and allows training on a dataset of any size. Prototype Memory consists of the limited-size memory module for storing recent class prototypes and employs a set of algorithms to update it in appropriate way. New class prototypes are generated on the fly using exemplar embeddings in the current mini-batch. These prototypes are enqueued to the memory and used in a role of classifier weights for softmax classification-based training. To prevent obsolescence and keep the memory in close connection with the encoder, prototypes are regularly refreshed, and oldest ones are dequeued and disposed of. Prototype Memory is computationally efficient and independent of dataset size. It can be used with various loss functions, hard example mining algorithms and encoder architectures. We prove the effectiveness of the proposed model by extensive experiments on popular face recognition benchmarks.