Multi-Dataset Benchmarks for Masked Identification using Contrastive Representation Learning

TL;DR

This paper introduces a new benchmark and a contrastive learning approach for masked face recognition, addressing the challenge of matching masked faces with unmasked reference images in security scenarios.

Contribution

It presents a set of re-purposed datasets, a benchmark, and a contrastive learning-based pre-training method specifically designed for masked face recognition.

Findings

Contrastive learning improves masked-unmasked face matching accuracy.

The proposed method outperforms standard face recognition features on various datasets.

Open-source tools and trained weights facilitate adoption in real-world systems.

Abstract

The COVID-19 pandemic has drastically changed accepted norms globally. Within the past year, masks have been used as a public health response to limit the spread of the virus. This sudden change has rendered many face recognition based access control, authentication and surveillance systems ineffective. Official documents such as passports, driving license and national identity cards are enrolled with fully uncovered face images. However, in the current global situation, face matching systems should be able to match these reference images with masked face images. As an example, in an airport or security checkpoint it is safer to match the unmasked image of the identifying document to the masked person rather than asking them to remove the mask. We find that current facial recognition techniques are not robust to this form of occlusion. To address this unique requirement presented due to the current circumstance, we propose a set of re-purposed datasets and a benchmark for researchers to use. We also propose a contrastive visual representation learning based pre-training workflow which is specialized to masked vs unmasked face matching. We ensure that our method learns robust features to differentiate people across varying data collection scenarios. We achieve this by training over many different datasets and validating our result by testing on various holdout datasets. The specialized weights trained by our method outperform standard face recognition features for masked to unmasked face matching. We believe the provided synthetic mask generating code, our novel training approach and the trained weights from the masked face models will help in adopting existing face recognition systems to operate in the current global environment. We open-source all contributions for broader use by the research community.