A Responsible Face Recognition Approach for Small and Mid-Scale Systems Through Personalized Neural Networks

TL;DR

This paper introduces MOTE, a personalized neural network approach for face recognition that enhances fairness and privacy in small to medium-scale systems by replacing traditional templates with dedicated classifiers.

Contribution

The paper proposes a novel model-template approach using small personalized neural networks, improving fairness and privacy in face recognition systems.

Findings

Significant fairness improvements demonstrated across multiple datasets.

Enhanced privacy protection compared to traditional template-based methods.

Trade-offs include increased inference time and storage requirements.

Abstract

Traditional face recognition systems rely on extracting fixed face representations, known as templates, to store and verify identities. These representations are typically generated by neural networks that often lack explainability and raise concerns regarding fairness and privacy. In this work, we propose a novel model-template (MOTE) approach that replaces vector-based face templates with small personalized neural networks. This design enables more responsible face recognition for small and medium-scale systems. During enrollment, MOTE creates a dedicated binary classifier for each identity, trained to determine whether an input face matches the enrolled identity. Each classifier is trained using only a single reference sample, along with synthetically balanced samples to allow adjusting fairness at the level of a single individual during enrollment. Extensive experiments across multiple datasets and recognition systems demonstrate substantial improvements in fairness and particularly in privacy. Although the method increases inference time and storage requirements, it presents a strong solution for small- and mid-scale applications where fairness and privacy are critical.