Conservative Plane Releasing for Spatial Privacy Protection in Mixed Reality

TL;DR

This paper introduces a spatial privacy protection method for AR/MR platforms using conservative data release and spatial generalization, effectively balancing user privacy with data utility in 3D spatial data sharing.

Contribution

It proposes a novel spatial privacy approach with generalized plane release and evaluates its effectiveness against recognition attacks in AR/MR environments.

Findings

Revealing no more than 11 generalized planes prevents adversary location identification in many cases.

Releasing up to 29 generalized planes maintains privacy while improving data utility.

The method effectively balances privacy and utility in 3D spatial data sharing for AR/MR.

Abstract

Augmented reality (AR) or mixed reality (MR) platforms require spatial understanding to detect objects or surfaces, often including their structural (i.e. spatial geometry) and photometric (e.g. color, and texture) attributes, to allow applications to place virtual or synthetic objects seemingly "anchored" on to real world objects; in some cases, even allowing interactions between the physical and virtual objects. These functionalities require AR/MR platforms to capture the 3D spatial information with high resolution and frequency; however, these pose unprecedented risks to user privacy. Aside from objects being detected, spatial information also reveals the location of the user with high specificity, e.g. in which part of the house the user is. In this work, we propose to leverage spatial generalizations coupled with conservative releasing to provide spatial privacy while maintaining data utility. We designed an adversary that builds up on existing place and shape recognition methods over 3D data as attackers to which the proposed spatial privacy approach can be evaluated against. Then, we simulate user movement within spaces which reveals more of their space as they move around utilizing 3D point clouds collected from Microsoft HoloLens. Results show that revealing no more than 11 generalized planes--accumulated from successively revealed spaces with large enough radius, i.e. $r\leq1.0m$--can make an adversary fail in identifying the spatial location of the user for at least half of the time. Furthermore, if the accumulated spaces are of smaller radius, i.e. each successively revealed space is $r\leq 0.5m$, we can release up to 29 generalized planes while enjoying both better data utility and privacy.