Towards Privacy-preserving Photorealistic Self-avatars in Mixed Reality

TL;DR

This paper introduces methods to create photorealistic MR avatars that preserve user identity while protecting biometric privacy, enabling broad social MR use without privacy risks.

Contribution

It proposes a novel approach to privatize avatar appearance by isolating identity in generative models and introduces two algorithms for identity obfuscation with privacy guarantees.

Findings

Successfully generates de-identified avatars in 2D and 3D.

Provides differential privacy guarantees for avatar identity.

Maintains realism and sense of self in privacy-preserving avatars.

Abstract

Photorealistic 3D avatar generation has rapidly improved in recent years, and realistic avatars that match a user's true appearance are more feasible in Mixed Reality (MR) than ever before. Yet, there are known risks to sharing one's likeness online, and photorealistic MR avatars could exacerbate these risks. If user likenesses were to be shared broadly, there are risks for cyber abuse or targeted fraud based on user appearances. We propose an alternate avatar rendering scheme for broader social MR -- synthesizing realistic avatars that preserve a user's demographic identity while being distinct enough from the individual user to protect facial biometric information. We introduce a methodology for privatizing appearance by isolating identity within the feature space of identity-encoding generative models. We develop two algorithms that then obfuscate identity: \epsmethod{} provides differential privacy guarantees and \thetamethod{} provides fine-grained control for the level of identity offset. These methods are shown to successfully generate de-identified virtual avatars across multiple generative architectures in 2D and 3D. With these techniques, it is possible to protect user privacy while largely preserving attributes related to sense of self. Employing these techniques in public settings could enable the use of photorealistic avatars broadly in MR, maintaining high realism and immersion without privacy risk.