A Quality-Guided Mixture of Score-Fusion Experts Framework for Human Recognition

TL;DR

This paper introduces QME, a novel quality-guided mixture of experts framework that enhances whole-body biometric recognition by learnably fusing scores from multiple modalities, addressing score distribution variations and improving accuracy.

Contribution

The paper proposes a new learnable score-fusion framework using a Mixture of Experts and introduces a pseudo-quality loss for better modality quality estimation.

Findings

Achieves state-of-the-art results on multiple datasets.

Effectively handles multimodal and multi-model variability.

Improves recognition performance by addressing score distribution variations.

Abstract

Whole-body biometric recognition is a challenging multimodal task that integrates various biometric modalities, including face, gait, and body. This integration is essential for overcoming the limitations of unimodal systems. Traditionally, whole-body recognition involves deploying different models to process multiple modalities, achieving the final outcome by score-fusion (e.g., weighted averaging of similarity matrices from each model). However, these conventional methods may overlook the variations in score distributions of individual modalities, making it challenging to improve final performance. In this work, we present \textbf{Q}uality-guided \textbf{M}ixture of score-fusion \textbf{E}xperts (QME), a novel framework designed for improving whole-body biometric recognition performance through a learnable score-fusion strategy using a Mixture of Experts (MoE). We introduce a novel pseudo-quality loss for quality estimation with a modality-specific Quality Estimator (QE), and a score triplet loss to improve the metric performance. Extensive experiments on multiple whole-body biometric datasets demonstrate the effectiveness of our proposed approach, achieving state-of-the-art results across various metrics compared to baseline methods. Our method is effective for multimodal and multi-model, addressing key challenges such as model misalignment in the similarity score domain and variability in data quality.