KVC-onGoing: Keystroke Verification Challenge

TL;DR

This paper introduces the KVC-onGoing challenge for benchmarking keystroke verification systems using large-scale, real-world data, demonstrating high discriminative power and addressing demographic fairness.

Contribution

It provides a standardized platform and protocol for benchmarking keystroke verification, with new state-of-the-art results on extensive public datasets.

Findings

Achieved low EER of around 3.3-3.6% in desktop and mobile scenarios.

Demonstrated high discriminative power of keystroke dynamics.

Identified demographic factors influencing verification scores.

Abstract

This article presents the Keystroke Verification Challenge - onGoing (KVC-onGoing), on which researchers can easily benchmark their systems in a common platform using large-scale public databases, the Aalto University Keystroke databases, and a standard experimental protocol. The keystroke data consist of tweet-long sequences of variable transcript text from over 185,000 subjects, acquired through desktop and mobile keyboards simulating real-life conditions. The results on the evaluation set of KVC-onGoing have proved the high discriminative power of keystroke dynamics, reaching values as low as 3.33% of Equal Error Rate (EER) and 11.96% of False Non-Match Rate (FNMR) @1% False Match Rate (FMR) in the desktop scenario, and 3.61% of EER and 17.44% of FNMR @1% at FMR in the mobile scenario, significantly improving previous state-of-the-art results. Concerning demographic fairness, the analyzed scores reflect the subjects' age and gender to various extents, not negligible in a few cases. The framework runs on CodaLab.