Partial FC: Training 10 Million Identities on a Single Machine

TL;DR

This paper introduces Partial FC, a method that enables training face recognition models with up to 10 million identities on a single machine by sampling classes and an efficient distributed sampling algorithm, reducing memory requirements.

Contribution

The paper proposes a novel class sampling strategy and an efficient distributed sampling algorithm that allow training with a vast number of identities on limited hardware.

Findings

Training with only 10% of classes does not reduce accuracy.

The method achieves classification with tens of millions of identities on eight GPUs.

Code is publicly available for reproducibility.

Abstract

Face recognition has been an active and vital topic among computer vision community for a long time. Previous researches mainly focus on loss functions used for facial feature extraction network, among which the improvements of softmax-based loss functions greatly promote the performance of face recognition. However, the contradiction between the drastically increasing number of face identities and the shortage of GPU memories is gradually becoming irreconcilable. In this paper, we thoroughly analyze the optimization goal of softmax-based loss functions and the difficulty of training massive identities. We find that the importance of negative classes in softmax function in face representation learning is not as high as we previously thought. The experiment demonstrates no loss of accuracy when training with only 10\% randomly sampled classes for the softmax-based loss functions, compared with training with full classes using state-of-the-art models on mainstream benchmarks. We also implement a very efficient distributed sampling algorithm, taking into account model accuracy and training efficiency, which uses only eight NVIDIA RTX2080Ti to complete classification tasks with tens of millions of identities. The code of this paper has been made available https://github.com/deepinsight/insightface/tree/master/recognition/partial_fc.