PaW-ViT: A Patch-based Warping Vision Transformer for Robust Ear Verification

TL;DR

PaW-ViT introduces a preprocessing technique based on anatomical knowledge to normalize ear images, improving the robustness of vision transformers against shape, size, and pose variations for biometric verification.

Contribution

It proposes a novel patch-based warping method that aligns ear features to natural curvature, enhancing transformer performance in biometric recognition tasks.

Findings

Improved robustness of ViT models to shape, size, and pose variations.

Effective alignment of ear features enhances token consistency.

Demonstrated success across multiple ViT architectures.

Abstract

The rectangular tokens common to vision transformer methods for visual recognition can strongly affect performance of these methods due to incorporation of information outside the objects to be recognized. This paper introduces PaW-ViT, Patch-based Warping Vision Transformer, a preprocessing approach rooted in anatomical knowledge that normalizes ear images to enhance the efficacy of ViT. By accurately aligning token boundaries to detected ear feature boundaries, PaW-ViT obtains greater robustness to shape, size, and pose variation. By aligning feature boundaries to natural ear curvature, it produces more consistent token representations for various morphologies. Experiments confirm the effectiveness of PaW-ViT on various ViT models (ViT-T, ViT-S, ViT-B, ViT-L) and yield reasonable alignment robustness to variation in shape, size, and pose. Our work aims to solve the disconnect between ear biometric morphological variation and transformer architecture positional sensitivity, presenting a possible avenue for authentication schemes.