TensorShield: Safeguarding On-Device Inference by Shielding Critical DNN Tensors with TEE

TL;DR

TensorShield introduces a novel method to protect critical tensors in on-device inference, balancing security and efficiency by shielding key model parts within TEEs, significantly reducing latency without sacrificing accuracy.

Contribution

It is the first work to shield partial tensors in TEEs for on-device inference, enhancing security while maintaining low latency and high accuracy.

Findings

Achieves near-complete security protection with partial tensor shielding.

Up to 25.35× faster inference compared to existing methods.

No accuracy loss during secure inference.

Abstract

To safeguard user data privacy, on-device inference has emerged as a prominent paradigm on mobile and Internet of Things (IoT) devices. This paradigm involves deploying a model provided by a third party on local devices to perform inference tasks. However, it exposes the private model to two primary security threats: model stealing (MS) and membership inference attacks (MIA). To mitigate these risks, existing wisdom deploys models within Trusted Execution Environments (TEEs), which is a secure isolated execution space. Nonetheless, the constrained secure memory capacity in TEEs makes it challenging to achieve full model security with low inference latency. This paper fills the gap with TensorShield, the first efficient on-device inference work that shields partial tensors of the model while still fully defending against MS and MIA. The key enabling techniques in TensorShield include: (i) a novel eXplainable AI (XAI) technique exploits the model's attention transition to assess critical tensors and shields them in TEE to achieve secure inference, and (ii) two meticulous designs with critical feature identification and latency-aware placement to accelerate inference while maintaining security. Extensive evaluations show that TensorShield delivers almost the same security protection as shielding the entire model inside TEE, while being up to 25.35$\times$ (avg. 5.85$\times$) faster than the state-of-the-art work, without accuracy loss.