BIG-MoE: Bypass Isolated Gating MoE for Generalized Multimodal Face Anti-Spoofing

TL;DR

This paper introduces BIG-MoE, a novel Mixture of Experts framework with isolated gating and differential prompt bypass, significantly enhancing multimodal face anti-spoofing by capturing subtle cues and resisting noise and overfitting.

Contribution

Proposes BIG-MoE, a new MoE model with isolated gating and differential prompt bypass to improve multimodal face anti-spoofing accuracy and robustness.

Findings

Achieves superior generalization on four benchmark datasets.

Effectively detects subtle spoofing cues with fine-grained experts.

Resists input noise and overfitting through isolation gating and prompt bypass.

Abstract

In the domain of facial recognition security, multimodal Face Anti-Spoofing (FAS) is essential for countering presentation attacks. However, existing technologies encounter challenges due to modality biases and imbalances, as well as domain shifts. Our research introduces a Mixture of Experts (MoE) model to address these issues effectively. We identified three limitations in traditional MoE approaches to multimodal FAS: (1) Coarse-grained experts' inability to capture nuanced spoofing indicators; (2) Gated networks' susceptibility to input noise affecting decision-making; (3) MoE's sensitivity to prompt tokens leading to overfitting with conventional learning methods. To mitigate these, we propose the Bypass Isolated Gating MoE (BIG-MoE) framework, featuring: (1) Fine-grained experts for enhanced detection of subtle spoofing cues; (2) An isolation gating mechanism to counteract input noise; (3) A novel differential convolutional prompt bypass enriching the gating network with critical local features, thereby improving perceptual capabilities. Extensive experiments on four benchmark datasets demonstrate significant generalization performance improvement in multimodal FAS task. The code is released at https://github.com/murInJ/BIG-MoE.