CryptoNite: Revealing the Pitfalls of End-to-End Private Inference at Scale

TL;DR

This paper critically evaluates current private inference protocols, revealing that high offline storage costs and online latency significantly hinder practicality, and proposes a modified protocol that improves end-to-end performance.

Contribution

It provides a comprehensive end-to-end analysis of private inference protocols, identifying key bottlenecks and introducing a modified protocol that reduces latency and storage costs.

Findings

Offline storage costs of garbled circuits are prohibitively high.

Current protocols have 10-1000× increased latency due to storage and computation bottlenecks.

The proposed protocol reduces mean PI latency by 4× for ResNet18 on TinyImageNet.

Abstract

The privacy concerns of providing deep learning inference as a service have underscored the need for private inference (PI) protocols that protect users' data and the service provider's model using cryptographic methods. Recently proposed PI protocols have achieved significant reductions in PI latency by moving the computationally heavy homomorphic encryption (HE) parts to an offline/pre-compute phase. Paired with recent optimizations that tailor networks for PI, these protocols have achieved performance levels that are tantalizingly close to being practical. In this paper, we conduct a rigorous end-to-end characterization of PI protocols and optimization techniques and find that the current understanding of PI performance is overly optimistic. Specifically, we find that offline storage costs of garbled circuits (GC), a key cryptographic protocol used in PI, on user/client devices are prohibitively high and force much of the expensive offline HE computation to the online phase, resulting in a 10-1000$\times$ increase to PI latency. We propose a modified PI protocol that significantly reduces client-side storage costs for a small increase in online latency. Evaluated end-to-end, the modified protocol outperforms current protocols by reducing the mean PI latency by $4\times$ for ResNet18 on TinyImageNet. We conclude with a discussion of several recently proposed PI optimizations in light of the findings and note many actually increase PI latency when evaluated from an end-to-end perspective.