Accelerating Private Large Transformers Inference through Fine-grained Collaborative Computation

TL;DR

FASTLMPI significantly accelerates private large transformer inference by optimizing fine-grained cryptographic computations, reducing runtime and communication costs while maintaining accuracy.

Contribution

The paper introduces FASTLMPI, a novel fine-grained co-design approach for homomorphic encryption and secret sharing to improve private transformer inference efficiency.

Findings

54% to 64% decrease in runtime

72.2% reduction in communication costs

Enhanced approximation accuracy for non-linear functions

Abstract

Homomorphic encryption (HE) and secret sharing (SS) enable computations on encrypted data, providing significant privacy benefits for large transformer-based models (TBM) in sensitive sectors like medicine and finance. However, private TBM inference incurs significant costs due to the coarse-grained application of HE and SS. We present FASTLMPI, a new approach to accelerate private TBM inference through fine-grained computation optimization. Specifically, through the fine-grained co-design of homomorphic encryption and secret sharing, FASTLMPI achieves efficient protocols for matrix multiplication, SoftMax, LayerNorm, and GeLU. In addition, FASTLMPI introduces a precise segmented approximation technique for differentiable non-linear, improving its fitting accuracy while maintaining a low polynomial degree. Compared to solution BOLT (S&P'24), FASTLMPI shows a remarkable 54% to 64% decrease in runtime and an impressive 72.2% reduction in communication costs.