MPC-Minimized Secure LLM Inference

TL;DR

Marill is a novel fine-tuning framework that reduces the computational and communication overhead of secure LLM inference using MPC, making privacy-preserving inference more practical without significant performance loss.

Contribution

It introduces architectural modifications during fine-tuning that minimize MPC operations, significantly improving efficiency of secure LLM inference.

Findings

3.6-11.3x faster runtime during secure inference

2.4-6.9x reduced communication overhead

Over 90% task performance preserved

Abstract

Many inference services based on large language models (LLMs) pose a privacy concern, either revealing user prompts to the service or the proprietary weights to the user. Secure inference offers a solution to this problem through secure multi-party computation (MPC), however, it is still impractical for modern LLM workload due to the large overhead imposed by MPC. To address this overhead, we propose Marill, a framework that adapts LLM fine-tuning to minimize MPC usage during secure inference. Marill introduces high-level architectural changes during fine-tuning that significantly reduce the number of expensive operations needed within MPC during inference, by removing some and relocating others outside MPC without compromising security. As a result, Marill-generated models are more efficient across all secure inference protocols and our approach complements MPC-friendly approximations for such operations. Compared to standard fine-tuning, Marill results in 3.6-11.3x better runtime and 2.4-6.9x better communication during secure inference across various MPC settings, while typically preserving over 90% performance across downstream tasks.