Privacy-Preserving Hierarchical Model-Distributed Inference

TL;DR

This paper presents a privacy-preserving hierarchical ML inference protocol that enhances speed and privacy by combining model parallelization, secret sharing, homomorphic encryption, and secure computation techniques, with minimal online communication.

Contribution

It introduces privateMDI, a novel hierarchical inference framework that reduces communication overhead and preserves privacy using advanced cryptographic methods and optimized offline/online phases.

Findings

Significantly reduces inference time compared to baselines.

Minimizes communication between clients, edge servers, and cloud.

Ensures privacy of data and model during inference.

Abstract

This paper focuses on designing a privacy-preserving Machine Learning (ML) inference protocol for a hierarchical setup, where clients own/generate data, model owners (cloud servers) have a pre-trained ML model, and edge servers perform ML inference on clients' data using the cloud server's ML model. Our goal is to speed up ML inference while providing privacy to both data and the ML model. Our approach (i) uses model-distributed inference (model parallelization) at the edge servers and (ii) reduces the amount of communication to/from the cloud server. Our privacy-preserving hierarchical model-distributed inference, privateMDI design uses additive secret sharing and linearly homomorphic encryption to handle linear calculations in the ML inference, and garbled circuit and a novel three-party oblivious transfer are used to handle non-linear functions. privateMDI consists of offline and online phases. We designed these phases in a way that most of the data exchange is done in the offline phase while the communication overhead of the online phase is reduced. In particular, there is no communication to/from the cloud server in the online phase, and the amount of communication between the client and edge servers is minimized. The experimental results demonstrate that privateMDI significantly reduces the ML inference time as compared to the baselines.