A Unified Framework for Masked and Mask-Free Face Recognition via Feature Rectification

TL;DR

This paper introduces FFR-Net, a unified face recognition framework that effectively recognizes both masked and mask-free faces by rectifying features to minimize differences, achieving state-of-the-art results especially under occlusion conditions.

Contribution

The paper proposes a novel unified framework with feature rectification blocks that align features from masked and mask-free faces, improving recognition accuracy under occlusion.

Findings

Achieves state-of-the-art performance on masked face recognition tasks.

Effectively minimizes feature discrepancies between masked and mask-free faces.

Demonstrates robustness in recognizing faces with occlusions like masks.

Abstract

Face recognition under ideal conditions is now considered a well-solved problem with advances in deep learning. Recognizing faces under occlusion, however, still remains a challenge. Existing techniques often fail to recognize faces with both the mouth and nose covered by a mask, which is now very common under the COVID-19 pandemic. Common approaches to tackle this problem include 1) discarding information from the masked regions during recognition and 2) restoring the masked regions before recognition. Very few works considered the consistency between features extracted from masked faces and from their mask-free counterparts. This resulted in models trained for recognizing masked faces often showing degraded performance on mask-free faces. In this paper, we propose a unified framework, named Face Feature Rectification Network (FFR-Net), for recognizing both masked and mask-free faces alike. We introduce rectification blocks to rectify features extracted by a state-of-the-art recognition model, in both spatial and channel dimensions, to minimize the distance between a masked face and its mask-free counterpart in the rectified feature space. Experiments show that our unified framework can learn a rectified feature space for recognizing both masked and mask-free faces effectively, achieving state-of-the-art results. Project code: https://github.com/haoosz/FFR-Net