Improving Acoustic Side-Channel Attacks on Keyboards Using Transformers and Large Language Models

TL;DR

This paper advances acoustic side-channel attacks on keyboards by integrating vision transformers and large language models, achieving state-of-the-art results and introducing noise mitigation and error correction techniques for real-world environments.

Contribution

It introduces the first application of transformers and language models to enhance ASCA effectiveness and robustness in noisy, real-world scenarios.

Findings

CoAtNet achieves 5.0% and 5.9% improvements over previous benchmarks.

Transformer models match CoAtNet's performance in keystroke classification.

LLMs effectively detect and correct keystroke errors in noisy environments.

Abstract

The increasing prevalence of microphones in everyday devices and the growing reliance on online services have amplified the risk of acoustic side-channel attacks (ASCAs) targeting keyboards. This study explores deep learning techniques, specifically vision transformers (VTs) and large language models (LLMs), to enhance the effectiveness and applicability of such attacks. We present substantial improvements over prior research, with the CoAtNet model achieving state-of-the-art performance. Our CoAtNet shows a 5.0% improvement for keystrokes recorded via smartphone (Phone) and 5.9% for those recorded via Zoom compared to previous benchmarks. We also evaluate transformer architectures and language models, with the best VT model matching CoAtNet's performance. A key advancement is the introduction of a noise mitigation method for real-world scenarios. By using LLMs for contextual understanding, we detect and correct erroneous keystrokes in noisy environments, enhancing ASCA performance. Additionally, fine-tuned lightweight language models with Low-Rank Adaptation (LoRA) deliver comparable performance to heavyweight models with 67X more parameters. This integration of VTs and LLMs improves the practical applicability of ASCA mitigation, marking the first use of these technologies to address ASCAs and error correction in real-world scenarios.