Falcon: Accelerating Homomorphically Encrypted Convolutions for Efficient Private Mobile Network Inference

TL;DR

Falcon introduces a novel packing algorithm that significantly accelerates homomorphic encryption-based private inference for mobile networks, improving latency and accuracy over previous methods.

Contribution

The paper presents Falcon, a dense packing algorithm with zero-aware greedy packing and communication-aware tiling, optimized for depthwise convolutions in HE-based 2PC frameworks.

Findings

Achieves over 15.6x latency reduction compared to CrypTFlow2.

Improves accuracy by 1.4% on CIFAR-100 and 4.2% on TinyImagenet.

Reduces inference overhead in HE-based private mobile network inference.

Abstract

Efficient networks, e.g., MobileNetV2, EfficientNet, etc, achieves state-of-the-art (SOTA) accuracy with lightweight computation. However, existing homomorphic encryption (HE)-based two-party computation (2PC) frameworks are not optimized for these networks and suffer from a high inference overhead. We observe the inefficiency mainly comes from the packing algorithm, which ignores the computation characteristics and the communication bottleneck of homomorphically encrypted depthwise convolutions. Therefore, in this paper, we propose Falcon, an effective dense packing algorithm for HE-based 2PC frameworks. Falcon features a zero-aware greedy packing algorithm and a communication-aware operator tiling strategy to improve the packing density for depthwise convolutions. Compared to SOTA HE-based 2PC frameworks, e.g., CrypTFlow2, Iron and Cheetah, Falcon achieves more than 15.6x, 5.1x and 1.8x latency reduction, respectively, at operator level. Meanwhile, at network level, Falcon allows for 1.4% and 4.2% accuracy improvement over Cheetah on CIFAR-100 and TinyImagenet datasets with iso-communication, respecitvely.