Advanced Acceptance Score: A Holistic Measure for Biometric Quantification

TL;DR

This paper introduces an advanced, holistic acceptance score for biometric evaluation that considers ranking, relevance, and disentanglement, providing a more comprehensive assessment than traditional error-rate-based measures.

Contribution

The paper proposes a novel holistic evaluation measure for biometric scores that incorporates ranking relevance, trend correspondence, and feature disentanglement, improving assessment accuracy.

Findings

The proposed measure outperforms existing evaluation metrics in identifying appropriate biometric scores.

It shows strong correlation with traditional measures, validating its reliability.

Experiments on three datasets with five models demonstrate its effectiveness.

Abstract

Quantifying biometric characteristics within hand gestures involve derivation of fitness scores from a gesture and identity aware feature space. However, evaluating the quality of these scores remains an open question. Existing biometric capacity estimation literature relies upon error rates. But these rates do not indicate goodness of scores. Thus, in this manuscript we present an exhaustive set of evaluation measures. We firstly identify ranking order and relevance of output scores as the primary basis for evaluation. In particular, we consider both rank deviation as well as rewards for: (i) higher scores of high ranked gestures and (ii) lower scores of low ranked gestures. We also compensate for correspondence between trends of output and ground truth scores. Finally, we account for disentanglement between identity features of gestures as a discounting factor. Integrating these elements with adequate weighting, we formulate advanced acceptance score as a holistic evaluation measure. To assess effectivity of the proposed we perform in-depth experimentation over three datasets with five state-of-the-art (SOTA) models. Results show that the optimal score selected with our measure is more appropriate than existing other measures. Also, our proposed measure depicts correlation with existing measures. This further validates its reliability. We have made our \href{https://github.com/AmanVerma2307/MeasureSuite}{code} public.