Towards Scalable and Privacy-Preserving Deep Neural Network via Algorithmic-Cryptographic Co-design

TL;DR

This paper introduces SPNN, a novel framework combining algorithmic graph splitting and cryptographic techniques like secret sharing and homomorphic encryption to enable scalable, privacy-preserving deep neural network training.

Contribution

It proposes a co-design approach that balances scalability and privacy by splitting DNN computations and applying cryptographic methods, with implementation and API support.

Findings

SPNN achieves better scalability than cryptography-only methods.

Experimental results show SPNN maintains high accuracy with privacy guarantees.

SPNN outperforms existing privacy-preserving DNN frameworks on real datasets.

Abstract

Deep Neural Networks (DNNs) have achieved remarkable progress in various real-world applications, especially when abundant training data are provided. However, data isolation has become a serious problem currently. Existing works build privacy preserving DNN models from either algorithmic perspective or cryptographic perspective. The former mainly splits the DNN computation graph between data holders or between data holders and server, which demonstrates good scalability but suffers from accuracy loss and potential privacy risks. In contrast, the latter leverages time-consuming cryptographic techniques, which has strong privacy guarantee but poor scalability. In this paper, we propose SPNN - a Scalable and Privacy-preserving deep Neural Network learning framework, from algorithmic-cryptographic co-perspective. From algorithmic perspective, we split the computation graph of DNN models into two parts, i.e., the private data related computations that are performed by data holders and the rest heavy computations that are delegated to a server with high computation ability. From cryptographic perspective, we propose using two types of cryptographic techniques, i.e., secret sharing and homomorphic encryption, for the isolated data holders to conduct private data related computations privately and cooperatively. Furthermore, we implement SPNN in a decentralized setting and introduce user-friendly APIs. Experimental results conducted on real-world datasets demonstrate the superiority of SPNN.