FaceLiVT: Face Recognition using Linear Vision Transformer with Structural Reparameterization For Mobile Device

TL;DR

FaceLiVT is a lightweight face recognition model combining CNN and Transformer architectures with a novel attention mechanism, achieving high accuracy and speed on mobile devices.

Contribution

The paper introduces FaceLiVT, a new hybrid CNN-Transformer model with a lightweight Multi-Head Linear Attention mechanism and reparameterized token mixer for efficient face recognition.

Findings

FaceLiVT outperforms state-of-the-art lightweight models on multiple benchmarks.

It achieves 8.6x faster inference than EdgeFace.

It is 21.2x faster than pure ViT models, with competitive accuracy.

Abstract

This paper introduces FaceLiVT, a lightweight yet powerful face recognition model that integrates a hybrid Convolution Neural Network (CNN)-Transformer architecture with an innovative and lightweight Multi-Head Linear Attention (MHLA) mechanism. By combining MHLA alongside a reparameterized token mixer, FaceLiVT effectively reduces computational complexity while preserving competitive accuracy. Extensive evaluations on challenging benchmarks; including LFW, CFP-FP, AgeDB-30, IJB-B, and IJB-C; highlight its superior performance compared to state-of-the-art lightweight models. MHLA notably improves inference speed, allowing FaceLiVT to deliver high accuracy with lower latency on mobile devices. Specifically, FaceLiVT is 8.6 faster than EdgeFace, a recent hybrid CNN-Transformer model optimized for edge devices, and 21.2 faster than a pure ViT-Based model. With its balanced design, FaceLiVT offers an efficient and practical solution for real-time face recognition on resource-constrained platforms.