Face Pyramid Vision Transformer

TL;DR

The paper introduces Face Pyramid Vision Transformer (FPVT), a new model that combines CNN and ViT features to improve face recognition accuracy while reducing computational costs.

Contribution

FPVT integrates novel modules like FSRA, FDR, IPE, and CFFN to enhance multi-scale facial feature learning and efficiency in face recognition tasks.

Findings

FPVT outperforms ten state-of-the-art methods on seven benchmarks.

FPVT achieves high accuracy with fewer parameters.

FPVT demonstrates robustness across diverse datasets.

Abstract

A novel Face Pyramid Vision Transformer (FPVT) is proposed to learn a discriminative multi-scale facial representations for face recognition and verification. In FPVT, Face Spatial Reduction Attention (FSRA) and Dimensionality Reduction (FDR) layers are employed to make the feature maps compact, thus reducing the computations. An Improved Patch Embedding (IPE) algorithm is proposed to exploit the benefits of CNNs in ViTs (e.g., shared weights, local context, and receptive fields) to model lower-level edges to higher-level semantic primitives. Within FPVT framework, a Convolutional Feed-Forward Network (CFFN) is proposed that extracts locality information to learn low level facial information. The proposed FPVT is evaluated on seven benchmark datasets and compared with ten existing state-of-the-art methods, including CNNs, pure ViTs, and Convolutional ViTs. Despite fewer parameters, FPVT has demonstrated excellent performance over the compared methods. Project page is available at https://khawar-islam.github.io/fpvt/