Modality Agnostic Heterogeneous Face Recognition with Switch Style Modulators

TL;DR

This paper presents a modality-agnostic heterogeneous face recognition framework using Switch Style Modulation Blocks that adaptively reduce domain gaps without requiring explicit target modality labels.

Contribution

The work introduces SSMB, a novel end-to-end trainable module that enables existing face recognition models to handle multiple modalities without explicit modality labels.

Findings

Effective cross-modal recognition demonstrated on benchmark datasets

Outperforms existing modality-specific HFR approaches

Seamless integration with pre-trained models

Abstract

Heterogeneous Face Recognition (HFR) systems aim to enhance the capability of face recognition in challenging cross-modal authentication scenarios. However, the significant domain gap between the source and target modalities poses a considerable challenge for cross-domain matching. Existing literature primarily focuses on developing HFR approaches for specific pairs of face modalities, necessitating the explicit training of models for each source-target combination. In this work, we introduce a novel framework designed to train a modality-agnostic HFR method capable of handling multiple modalities during inference, all without explicit knowledge of the target modality labels. We achieve this by implementing a computationally efficient automatic routing mechanism called Switch Style Modulation Blocks (SSMB) that trains various domain expert modulators which transform the feature maps adaptively reducing the domain gap. Our proposed SSMB can be trained end-to-end and seamlessly integrated into pre-trained face recognition models, transforming them into modality-agnostic HFR models. We have performed extensive evaluations on HFR benchmark datasets to demonstrate its effectiveness. The source code and protocols will be made publicly available.