Artemis: HE-Aware Training for Efficient Privacy-Preserving Machine Learning

TL;DR

Artemis introduces a novel HE-aware pruning method for deep neural networks that significantly reduces computational costs in privacy-preserving ML, enabling more practical HE-based inference.

Contribution

It proposes a new DNN pruning technique tailored for HE-PPML, coupling training with group Lasso regularization to optimize HE-specific cost reduction.

Findings

Achieves 1.2-6x improvement in inference efficiency on ResNet models.

Identifies diagonal pruning as the Pareto-optimal strategy for HE-aware pruning.

Outperforms prior HE-oriented pruning methods.

Abstract

Privacy-Preserving ML (PPML) based on Homomorphic Encryption (HE) is a promising foundational privacy technology. Making it more practical requires lowering its computational cost, especially, in handling modern large deep neural networks. Model compression via pruning is highly effective in conventional plaintext ML but cannot be effectively applied to HE-PPML as is. We propose Artemis, a highly effective DNN pruning technique for HE-based inference. We judiciously investigate two HE-aware pruning strategies (positional and diagonal) to reduce the number of Rotation operations, which dominate compute time in HE convolution. We find that Pareto-optimal solutions are based fully on diagonal pruning. Artemis' benefits come from coupling DNN training, driven by a novel group Lasso regularization objective, with pruning to maximize HE-specific cost reduction (dominated by the Rotation operations). We show that Artemis improves on prior HE-oriented pruning and can achieve a 1.2-6x improvement when targeting modern convolutional models (ResNet18 and ResNet18) across three datasets.