Siren Song: Manipulating Pose Estimation in XR Headsets Using Acoustic Attacks

TL;DR

This paper demonstrates that acoustic signals can manipulate pose estimation in XR headsets by interfering with IMU sensors, leading to security vulnerabilities such as input manipulation and denial of service.

Contribution

The study introduces novel acoustic attack methods on XR headsets, revealing vulnerabilities in pose estimation systems and demonstrating practical attack scenarios.

Findings

Acoustic signals can induce false IMU readings in XR headsets.

Attacks can manipulate user input and cause denial of service.

Commercial XR devices are vulnerable to acoustic interference.

Abstract

Extended Reality (XR) experiences involve interactions between users, the real world, and virtual content. A key step to enable these experiences is the XR headset sensing and estimating the user's pose in order to accurately place and render virtual content in the real world. XR headsets use multiple sensors (e.g., cameras, inertial measurement unit) to perform pose estimation and improve its robustness, but this provides an attack surface for adversaries to interfere with the pose estimation process. In this paper, we create and study the effects of acoustic attacks that create false signals in the inertial measurement unit (IMU) on XR headsets, leading to adverse downstream effects on XR applications. We generate resonant acoustic signals on a HoloLens 2 and measure the resulting perturbations in the IMU readings, and also demonstrate both fine-grained and coarse attacks on the popular ORB-SLAM3 and an open-source XR system (ILLIXR). With the knowledge gleaned from attacking these open-source frameworks, we demonstrate four end-to-end proof-of-concept attacks on a HoloLens 2: manipulating user input, clickjacking, zone invasion, and denial of user interaction. Our experiments show that current commercial XR headsets are susceptible to acoustic attacks, raising concerns for their security.