Privacy-Preserving Inference for Quantized BERT Models

TL;DR

This paper introduces a privacy-preserving inference method for quantized BERT models that significantly reduces computational overhead using layer-wise quantization, secure lookup tables, and dual secret sharing, enabling efficient secure NLP inference.

Contribution

It proposes a novel layer-wise quantization scheme, supports 1-bit weights in secure inference, and designs efficient protocols for nonlinear functions like softmax, improving speed and privacy in secure BERT inference.

Findings

Achieves up to 22x speedup over previous methods.

Supports 1-bit weight fully connected layers securely.

Reduces overhead in privacy-preserving BERT inference.

Abstract

With the increasing deployment of generative machine learning models in privacy-sensitive domains such as healthcare and personalized services, ensuring secure inference has become a critical challenge. Secure multi-party computation (MPC) enables privacy-preserving model inference but suffers from high communication and computation overhead. The main bottleneck lies in the expensive secure evaluation of floating-point operations. Quantization offers a promising solution by converting floating-point operations into lower-precision integer computations, significantly reducing overhead. However, existing MPC-based quantized inference methods either rely on public quantization parameters-posing privacy risks-or suffer from inefficiencies, particularly in handling nonlinear functions such as activations and softmax. In this work, we propose a fine-grained, layer-wise quantization scheme and support 1-bit weight fully connected layers in a secure setting. We design a multi-input lookup table protocol to evaluate softmax efficiently and securely. Furthermore, we use dual secret sharing schemes and perform precision conversions via lookup tables, eliminating truncation overhead entirely. Experimental evaluation on BERT-base models demonstrates that our approach achieves up to $8\times$ speedup compared to Lu \emph{et al}. (NDSS 25), $9\times$ speedup compared to Gupta \emph{et al}. (PETS 24) and $22 \times$ speedup compared to Knott \emph{et al}. (NeurIPS 21).