FedFR: Joint Optimization Federated Framework for Generic and Personalized Face Recognition

TL;DR

FedFR is a federated learning framework that enhances both generic and personalized face recognition models while preserving user privacy, marking the first exploration of personalized face recognition within federated learning setups.

Contribution

Introduces FedFR, a novel federated learning framework that jointly optimizes generic and personalized face recognition models, addressing privacy concerns and personalization needs.

Findings

Outperforms previous methods on multiple face recognition benchmarks.

Effectively balances generic and personalized face recognition in federated settings.

First to explore personalized face recognition within federated learning.

Abstract

Current state-of-the-art deep learning based face recognition (FR) models require a large number of face identities for central training. However, due to the growing privacy awareness, it is prohibited to access the face images on user devices to continually improve face recognition models. Federated Learning (FL) is a technique to address the privacy issue, which can collaboratively optimize the model without sharing the data between clients. In this work, we propose a FL based framework called FedFR to improve the generic face representation in a privacy-aware manner. Besides, the framework jointly optimizes personalized models for the corresponding clients via the proposed Decoupled Feature Customization module. The client-specific personalized model can serve the need of optimized face recognition experience for registered identities at the local device. To the best of our knowledge, we are the first to explore the personalized face recognition in FL setup. The proposed framework is validated to be superior to previous approaches on several generic and personalized face recognition benchmarks with diverse FL scenarios. The source codes and our proposed personalized FR benchmark under FL setup are available at https://github.com/jackie840129/FedFR.