Domain Private and Agnostic Feature for Modality Adaptive Face Recognition

TL;DR

This paper introduces a novel feature aggregation network that disentangles domain-private and domain-agnostic features for improved modality adaptive face recognition, effectively handling modality discrepancies and dataset imbalance.

Contribution

The work proposes a new FAN model with disentangled representation, feature fusion, and adaptive penalty metric learning, advancing cross-modal face recognition by better separating identity and modality features.

Findings

FAN outperforms state-of-the-art methods on benchmark datasets.

The disentangled features improve cross-modal recognition accuracy.

Adaptive penalty metric enhances intra-class compactness and inter-class separation.

Abstract

Heterogeneous face recognition is a challenging task due to the large modality discrepancy and insufficient cross-modal samples. Most existing works focus on discriminative feature transformation, metric learning and cross-modal face synthesis. However, the fact that cross-modal faces are always coupled by domain (modality) and identity information has received little attention. Therefore, how to learn and utilize the domain-private feature and domain-agnostic feature for modality adaptive face recognition is the focus of this work. Specifically, this paper proposes a Feature Aggregation Network (FAN), which includes disentangled representation module (DRM), feature fusion module (FFM) and adaptive penalty metric (APM) learning session. First, in DRM, two subnetworks, i.e. domain-private network and domain-agnostic network are specially designed for learning modality features and identity features, respectively. Second, in FFM, the identity features are fused with domain features to achieve cross-modal bi-directional identity feature transformation, which, to a large extent, further disentangles the modality information and identity information. Third, considering that the distribution imbalance between easy and hard pairs exists in cross-modal datasets, which increases the risk of model bias, the identity preserving guided metric learning with adaptive hard pairs penalization is proposed in our FAN. The proposed APM also guarantees the cross-modality intra-class compactness and inter-class separation. Extensive experiments on benchmark cross-modal face datasets show that our FAN outperforms SOTA methods.