Comet: A Communication-efficient and Performant Approximation for Private Transformer Inference

TL;DR

This paper introduces Comet, a novel method that significantly reduces communication costs in private Transformer inference, achieving faster performance with minimal accuracy loss.

Contribution

Comet is a new plug-in approach that effectively cuts communication overhead and speeds up private Transformer inference without sacrificing model accuracy.

Findings

Up to 3.9× less communication required

Achieves 3.5× speedups in inference

Maintains competitive performance on GLUE benchmarks

Abstract

The prevalent use of Transformer-like models, exemplified by ChatGPT in modern language processing applications, underscores the critical need for enabling private inference essential for many cloud-based services reliant on such models. However, current privacy-preserving frameworks impose significant communication burden, especially for non-linear computation in Transformer model. In this paper, we introduce a novel plug-in method Comet to effectively reduce the communication cost without compromising the inference performance. We second introduce an efficient approximation method to eliminate the heavy communication in finding good initial approximation. We evaluate our Comet on Bert and RoBERTa models with GLUE benchmark datasets, showing up to 3.9$\times$ less communication and 3.5$\times$ speedups while keep competitive model performance compared to the prior art.