Combined Learning of Salient Local Descriptors and Distance Metrics for Image Set Face Verification

TL;DR

This paper introduces a novel face verification method that combines local feature descriptors with learned distance metrics, improving robustness and accuracy in image set matching across various challenging conditions.

Contribution

It proposes a new approach that jointly learns discriminative face regions and optimal metric combinations, enhancing image set face verification performance.

Findings

Outperforms state-of-the-art methods on LFW, PIE, and MOBIO datasets.

Utilizes local features inspired by human visual hierarchy for robustness.

Achieves significant accuracy improvements in face verification tasks.

Abstract

In contrast to comparing faces via single exemplars, matching sets of face images increases robustness and discrimination performance. Recent image set matching approaches typically measure similarities between subspaces or manifolds, while representing faces in a rigid and holistic manner. Such representations are easily affected by variations in terms of alignment, illumination, pose and expression. While local feature based representations are considerably more robust to such variations, they have received little attention within the image set matching area. We propose a novel image set matching technique, comprised of three aspects: (i) robust descriptors of face regions based on local features, partly inspired by the hierarchy in the human visual system, (ii) use of several subspace and exemplar metrics to compare corresponding face regions, (iii) jointly learning which regions are the most discriminative while finding the optimal mixing weights for combining metrics. Face recognition experiments on LFW, PIE and MOBIO face datasets show that the proposed algorithm obtains considerably better performance than several recent state-of-the-art techniques, such as Local Principal Angle and the Kernel Affine Hull Method.