Level Playing Field for Million Scale Face Recognition

TL;DR

This paper introduces MF2, a large-scale, standardized benchmark for face recognition with 672K identities, revealing that algorithms trained on it can achieve state-of-the-art results comparable to private datasets.

Contribution

The creation of MF2, a public, large-scale benchmark dataset for fair comparison of face recognition algorithms at million scale.

Findings

Algorithms trained on MF2 achieve state-of-the-art results.

Some algorithms outperform their previous results when trained on MF2.

Age invariance remains a challenge, indicating need for larger age variation data.

Abstract

Face recognition has the perception of a solved problem, however when tested at the million-scale exhibits dramatic variation in accuracies across the different algorithms. Are the algorithms very different? Is access to good/big training data their secret weapon? Where should face recognition improve? To address those questions, we created a benchmark, MF2, that requires all algorithms to be trained on same data, and tested at the million scale. MF2 is a public large-scale set with 672K identities and 4.7M photos created with the goal to level playing field for large scale face recognition. We contrast our results with findings from the other two large-scale benchmarks MegaFace Challenge and MS-Celebs-1M where groups were allowed to train on any private/public/big/small set. Some key discoveries: 1) algorithms, trained on MF2, were able to achieve state of the art and comparable results to algorithms trained on massive private sets, 2) some outperformed themselves once trained on MF2, 3) invariance to aging suffers from low accuracies as in MegaFace, identifying the need for larger age variations possibly within identities or adjustment of algorithms in future testings.

Figures

Full text

See pages 1-last of 3276.pdf