Combined CNN and ViT features off-the-shelf: Another astounding baseline for recognition

TL;DR

This paper demonstrates that off-the-shelf CNN and ViT features, combined with traditional features, provide a highly effective and resource-efficient baseline for periocular recognition, outperforming many existing methods.

Contribution

It extends previous work by incorporating Vision Transformers with CNNs for periocular recognition, showing their complementarity and efficiency in resource-limited settings.

Findings

CNNs and ViTs are highly complementary for recognition.

Small pre-trained models can achieve high accuracy.

Combining features improves recognition performance.

Abstract

We apply pre-trained architectures, originally developed for the ImageNet Large Scale Visual Recognition Challenge, for periocular recognition. These architectures have demonstrated significant success in various computer vision tasks beyond the ones for which they were designed. This work builds on our previous study using off-the-shelf Convolutional Neural Network (CNN) and extends it to include the more recently proposed Vision Transformers (ViT). Despite being trained for generic object classification, middle-layer features from CNNs and ViTs are a suitable way to recognize individuals based on periocular images. We also demonstrate that CNNs and ViTs are highly complementary since their combination results in boosted accuracy. In addition, we show that a small portion of these pre-trained models can achieve good accuracy, resulting in thinner models with fewer parameters, suitable for resource-limited environments such as mobiles. This efficiency improves if traditional handcrafted features are added as well.