S3: Side-Channel Attack on Stylus Pencil through Sensors

TL;DR

This paper demonstrates a side-channel attack that infers what a user writes on an iPad using only motion sensor data, revealing potential privacy risks in smart device sensor access.

Contribution

The paper introduces S3, a novel system that leverages motion sensors and magnetic data to accurately identify user input on an iPad with an Apple Pencil, highlighting new security vulnerabilities.

Findings

Achieves over 93% accuracy in identifying letters and words

Uses a high-dimensional particle filter for tracking Pencil movements

Builds a magnetic map to correlate magnetic data with Pencil position

Abstract

With smart devices being an essential part of our everyday lives, unsupervised access to the mobile sensors' data can result in a multitude of side-channel attacks. In this paper, we study potential data leaks from Apple Pencil (2nd generation) supported by the Apple iPad Pro, the latest stylus pen which attaches to the iPad body magnetically for charging. We observe that the Pencil's body affects the magnetic readings sensed by the iPad's magnetometer when a user is using the Pencil. Therefore, we ask: Can we infer what a user is writing on the iPad screen with the Apple Pencil, given access to only the iPad's motion sensors' data? To answer this question, we present Side-channel attack on Stylus pencil through Sensors (S3), a system that identifies what a user is writing from motion sensor readings. We first use the sharp fluctuations in the motion sensors' data to determine when a user is writing on the iPad. We then introduce a high-dimensional particle filter to track the location and orientation of the Pencil during usage. Lastly, to guide particles, we build the Pencil's magnetic map serving as a bridge between the measured magnetic data and the Pencil location and orientation. We evaluate S3 with 10 subjects and demonstrate that we correctly identify 93.9%, 96%, 97.9%, and 93.33% of the letters, numbers, shapes, and words by only having access to the motion sensors' data.