La-SoftMoE CLIP for Unified Physical-Digital Face Attack Detection

TL;DR

This paper introduces La-SoftMoE CLIP, a novel model that effectively detects both physical and digital face attacks by adaptively handling diverse attack data within a unified framework, achieving state-of-the-art results.

Contribution

The paper presents a new La-SoftMoE CLIP model that uses a flexible self-adapting weighting mechanism within a Mixture of Experts framework for unified face attack detection.

Findings

Achieves state-of-the-art performance on attack detection tasks.

Effectively handles diverse physical and digital attack data.

Demonstrates superior adaptability and accuracy over existing methods.

Abstract

Facial recognition systems are susceptible to both physical and digital attacks, posing significant security risks. Traditional approaches often treat these two attack types separately due to their distinct characteristics. Thus, when being combined attacked, almost all methods could not deal. Some studies attempt to combine the sparse data from both types of attacks into a single dataset and try to find a common feature space, which is often impractical due to the space is difficult to be found or even non-existent. To overcome these challenges, we propose a novel approach that uses the sparse model to handle sparse data, utilizing different parameter groups to process distinct regions of the sparse feature space. Specifically, we employ the Mixture of Experts (MoE) framework in our model, expert parameters are matched to tokens with varying weights during training and adaptively activated during testing. However, the traditional MoE struggles with the complex and irregular classification boundaries of this problem. Thus, we introduce a flexible self-adapting weighting mechanism, enabling the model to better fit and adapt. In this paper, we proposed La-SoftMoE CLIP, which allows for more flexible adaptation to the Unified Attack Detection (UAD) task, significantly enhancing the model's capability to handle diversity attacks. Experiment results demonstrate that our proposed method has SOTA performance.