Approximating ReLU on a Reduced Ring for Efficient MPC-based Private Inference

TL;DR

HummingBird is an MPC framework that significantly reduces communication overhead in private inference by selectively discarding bits during ReLU evaluation, achieving over 2x speedup with minimal accuracy loss.

Contribution

It introduces a novel bit-discarding method for ReLU in MPC, reducing communication by up to 8.76x while maintaining high accuracy.

Findings

Achieves 2.03--2.67x end-to-end speedup in MPC inference.

Reduces ReLU communication by up to 8.76x with minimal accuracy loss.

Discards 87--91% of bits during ReLU evaluation.

Abstract

Secure multi-party computation (MPC) allows users to offload machine learning inference on untrusted servers without having to share their privacy-sensitive data. Despite their strong security properties, MPC-based private inference has not been widely adopted in the real world due to their high communication overhead. When evaluating ReLU layers, MPC protocols incur a significant amount of communication between the parties, making the end-to-end execution time multiple orders slower than its non-private counterpart. This paper presents HummingBird, an MPC framework that reduces the ReLU communication overhead significantly by using only a subset of the bits to evaluate ReLU on a smaller ring. Based on theoretical analyses, HummingBird identifies bits in the secret share that are not crucial for accuracy and excludes them during ReLU evaluation to reduce communication. With its efficient search engine, HummingBird discards 87--91% of the bits during ReLU and still maintains high accuracy. On a real MPC setup involving multiple servers, HummingBird achieves on average 2.03--2.67x end-to-end speedup without introducing any errors, and up to 8.64x average speedup when some amount of accuracy degradation can be tolerated, due to its up to 8.76x communication reduction.