Towards robustness under occlusion for face recognition

TL;DR

This paper investigates how occlusions affect face recognition performance and proposes two methods, inpainting and Cutmix, to improve robustness, demonstrating that these strategies effectively mitigate performance drops caused by occlusions.

Contribution

The study introduces and compares two novel approaches, inpainting and Cutmix, to enhance face recognition robustness under occlusion conditions.

Findings

Cutmix improves robustness without extra application steps.

Both inpainting and Cutmix significantly reduce error rates under occlusion.

Public datasets with occlusion masks and inpainted images are released.

Abstract

In this paper, we evaluate the effects of occlusions in the performance of a face recognition pipeline that uses a ResNet backbone. The classifier was trained on a subset of the CelebA-HQ dataset containing 5,478 images from 307 classes, to achieve top-1 error rate of 17.91%. We designed 8 different occlusion masks which were applied to the input images. This caused a significant drop in the classifier performance: its error rate for each mask became at least two times worse than before. In order to increase robustness under occlusions, we followed two approaches. The first is image inpainting using the pre-trained pluralistic image completion network. The second is Cutmix, a regularization strategy consisting of mixing training images and their labels using rectangular patches, making the classifier more robust against input corruptions. Both strategies revealed effective and interesting results were observed. In particular, the Cutmix approach makes the network more robust without requiring additional steps at the application time, though its training time is considerably longer. Our datasets containing the different occlusion masks as well as their inpainted counterparts are made publicly available to promote research on the field.