Crypto-Oriented Neural Architecture Design

TL;DR

This paper introduces a novel neural network architecture design optimized for secure computation, featuring a new Partial Activation layer that significantly enhances the efficiency of privacy-preserving inference.

Contribution

It proposes a new crypto-oriented neural architecture design with a Partial Activation layer, improving secure inference efficiency across multiple models.

Findings

Significant speedup in secure inference performance

Effective optimization of neural architecture for cryptographic tasks

Demonstrated improvements on SqueezeNet, ShuffleNetV2, and MobileNetV2

Abstract

As neural networks revolutionize many applications, significant privacy conflicts between model users and providers emerge. The cryptography community developed a variety of techniques for secure computation to address such privacy issues. As generic techniques for secure computation are typically prohibitively ineffective, many efforts focus on optimizing their underlying cryptographic tools. Differently, we propose to optimize the initial design of crypto-oriented neural architectures and provide a novel Partial Activation layer. The proposed layer is much faster for secure computation. Evaluating our method on three state-of-the-art architectures (SqueezeNet, ShuffleNetV2, and MobileNetV2) demonstrates significant improvement to the efficiency of secure inference on common evaluation metrics.