G$^2$V$^2$former: Graph Guided Video Vision Transformer for Face Anti-Spoofing

TL;DR

This paper introduces G$^2$V$^2$former, a novel graph-guided video vision transformer that fuses photometric and dynamic facial features for improved face anti-spoofing, capturing both spatial and temporal cues effectively.

Contribution

It proposes a new spatiotemporal attention mechanism with Kronecker temporal attention and leverages facial landmarks to enhance dynamic feature extraction in face anti-spoofing.

Findings

Achieves superior performance on nine benchmark datasets.

Effectively captures dynamic and photometric cues for spoofing detection.

Outperforms existing methods in various scenarios.

Abstract

In videos containing spoofed faces, we may uncover the spoofing evidence based on either photometric or dynamic abnormality, even a combination of both. Prevailing face anti-spoofing (FAS) approaches generally concentrate on the single-frame scenario, however, purely photometric-driven methods overlook the dynamic spoofing clues that may be exposed over time. This may lead FAS systems to conclude incorrect judgments, especially in cases where it is easily distinguishable in terms of dynamics but challenging to discern in terms of photometrics. To this end, we propose the Graph Guided Video Vision Transformer (G$^2$V$^2$former), which combines faces with facial landmarks for photometric and dynamic feature fusion. We factorize the attention into space and time, and fuse them via a spatiotemporal block. Specifically, we design a novel temporal attention called Kronecker temporal attention, which has a wider receptive field, and is beneficial for capturing dynamic information. Moreover, we leverage the low-semantic motion of facial landmarks to guide the high-semantic change of facial expressions based on the motivation that regions containing landmarks may reveal more dynamic clues. Extensive experiments on nine benchmark datasets demonstrate that our method achieves superior performance under various scenarios. The codes will be released soon.