TempCharBERT: Keystroke Dynamics for Continuous Access Control Based on Pre-trained Language Models

TL;DR

This paper introduces TempCharBERT, a novel model that integrates temporal keystroke data into pre-trained language models to improve continuous user authentication and identification, demonstrating significant accuracy gains and privacy-preserving training.

Contribution

The paper proposes TempCharBERT, a new architecture that incorporates temporal keystroke features into CharBERT, enabling effective user recognition and authentication from keystroke dynamics.

Findings

TempCharBERT significantly outperforms baseline models in user identification accuracy.

The model can be trained effectively in federated learning environments.

Incorporating temporal information enhances keystroke-based authentication performance.

Abstract

With the widespread of digital environments, reliable authentication and continuous access control has become crucial. It can minimize cyber attacks and prevent frauds, specially those associated with identity theft. A particular interest lies on keystroke dynamics (KD), which refers to the task of recognizing individuals' identity based on their unique typing style. In this work, we propose the use of pre-trained language models (PLMs) to recognize such patterns. Although PLMs have shown high performance on multiple NLP benchmarks, the use of these models on specific tasks requires customization. BERT and RoBERTa, for instance, rely on subword tokenization, and they cannot be directly applied to KD, which requires temporal-character information to recognize users. Recent character-aware PLMs are able to process both subwords and character-level information and can be an alternative solution. Notwithstanding, they are still not suitable to be directly fine-tuned for KD as they are not optimized to account for user's temporal typing information (e.g., hold time and flight time). To overcome this limitation, we propose TempCharBERT, an architecture that incorporates temporal-character information in the embedding layer of CharBERT. This allows modeling keystroke dynamics for the purpose of user identification and authentication. Our results show a significant improvement with this customization. We also showed the feasibility of training TempCharBERT on a federated learning settings in order to foster data privacy.