EarSpy: Spying Caller Speech and Identity through Tiny Vibrations of Smartphone Ear Speakers

TL;DR

This paper demonstrates that tiny vibrations from smartphone ear speakers can be captured by motion sensors to eavesdrop on private speech, revealing significant privacy risks with high accuracy in speaker and gender detection.

Contribution

The study shows the feasibility of using built-in motion sensors to eavesdrop on speech through ear speakers, a previously believed impossible attack vector.

Findings

Achieved up to 98.66% accuracy in gender detection

Achieved 92.6% accuracy in speaker detection

Achieved 56.42% accuracy in digit detection

Abstract

Eavesdropping from the user's smartphone is a well-known threat to the user's safety and privacy. Existing studies show that loudspeaker reverberation can inject speech into motion sensor readings, leading to speech eavesdropping. While more devastating attacks on ear speakers, which produce much smaller scale vibrations, were believed impossible to eavesdrop with zero-permission motion sensors. In this work, we revisit this important line of reach. We explore recent trends in smartphone manufacturers that include extra/powerful speakers in place of small ear speakers, and demonstrate the feasibility of using motion sensors to capture such tiny speech vibrations. We investigate the impacts of these new ear speakers on built-in motion sensors and examine the potential to elicit private speech information from the minute vibrations. Our designed system EarSpy can successfully detect word regions, time, and frequency domain features and generate a spectrogram for each word region. We train and test the extracted data using classical machine learning algorithms and convolutional neural networks. We found up to 98.66% accuracy in gender detection, 92.6% detection in speaker detection, and 56.42% detection in digit detection (which is 5X more significant than the random selection (10%)). Our result unveils the potential threat of eavesdropping on phone conversations from ear speakers using motion sensors.