Impact of Data Breadth and Depth on Performance of Siamese Neural Network Model: Experiments with Three Keystroke Dynamic Datasets

TL;DR

This study investigates how the number of subjects and samples per subject in keystroke datasets affect the performance of Siamese Neural Networks, revealing that dataset breadth generally improves model effectiveness, while depth effects vary by dataset type.

Contribution

The paper provides a comprehensive experimental analysis of dataset breadth and depth impacts on SNN performance across three keystroke datasets, offering new insights for biometric system design.

Findings

Increasing dataset breadth improves inter-subject variability capture.

Depth effects depend on dataset type; more impact on free-text datasets.

Fixed-text datasets are less sensitive to depth variations.

Abstract

Deep learning models, such as the Siamese Neural Networks (SNN), have shown great potential in capturing the intricate patterns in behavioral data. However, the impacts of dataset breadth (i.e., the number of subjects) and depth (e.g., the amount of training samples per subject) on the performance of these models is often informally assumed, and remains under-explored. To this end, we have conducted extensive experiments using the concepts of "feature space" and "density" to guide and gain deeper understanding on the impact of dataset breadth and depth on three publicly available keystroke datasets (Aalto, CMU and Clarkson II). Through varying the number of training subjects, number of samples per subject, amount of data in each sample, and number of triplets used in training, we found that when feasible, increasing dataset breadth enables the training of a well-trained model that effectively captures more inter-subject variability. In contrast, we find that the extent of depth's impact from a dataset depends on the nature of the dataset. Free-text datasets are influenced by all three depth-wise factors; inadequate samples per subject, sequence length, training triplets and gallery sample size, which may all lead to an under-trained model. Fixed-text datasets are less affected by these factors, and as such make it easier to create a well-trained model. These findings shed light on the importance of dataset breadth and depth in training deep learning models for behavioral biometrics and provide valuable insights for designing more effective authentication systems.