Hey Alexa what did I just type? Decoding smartphone sounds with a voice assistant

TL;DR

This paper demonstrates that voice assistants can be exploited to extract sensitive data like PINs and messages from nearby smartphones, highlighting new privacy risks beyond spoken conversations.

Contribution

It reveals that remote keyboard-inference attacks can target virtual keyboards via voice assistant recordings, extending privacy concerns to smartphone typing activities.

Findings

Attacker can extract PINs and messages from recordings

Attacks work up to half a meter away

Remote inference extends beyond physical keyboards

Abstract

Voice assistants are now ubiquitous and listen in on our everyday lives. Ever since they became commercially available, privacy advocates worried that the data they collect can be abused: might private conversations be extracted by third parties? In this paper we show that privacy threats go beyond spoken conversations and include sensitive data typed on nearby smartphones. Using two different smartphones and a tablet we demonstrate that the attacker can extract PIN codes and text messages from recordings collected by a voice assistant located up to half a meter away. This shows that remote keyboard-inference attacks are not limited to physical keyboards but extend to virtual keyboards too. As our homes become full of always-on microphones, we need to work through the implications.