Model-based Large Language Model Customization as Service

TL;DR

Llamdex enables privacy-preserving large language model customization by allowing clients to upload protected domain-specific models instead of data, improving accuracy without compromising inference efficiency.

Contribution

Introduces Llamdex, a framework for LLM customization that uses client-uploaded models and connection modules trained without sensitive data, enhancing privacy and performance.

Findings

Improves domain-specific accuracy by up to 26% over state-of-the-art private synthesis methods.

Maintains inference efficiency comparable to original LLM services.

Reduces privacy risks by avoiding data upload for fine-tuning.

Abstract

Prominent Large Language Model (LLM) services from providers like OpenAI and Google excel at general tasks but often underperform on domain-specific applications. Current customization services for these LLMs typically require users to upload data for fine-tuning, posing significant privacy risks. While differentially private (DP) data synthesis presents a potential alternative, its application commonly results in low effectiveness due to the introduction of excessive noise on data for DP. To overcome this, we introduce Llamdex, a novel framework that facilitates LLM customization as a service, where the client uploads pre-trained domain-specific models rather than data. This client-uploaded model, optionally protected by DP with much lower noise, is inserted into the base LLM via connection modules. Significantly, these connecting modules are trained without requiring sensitive domain data, enabling clients to customize LLM services while preserving data privacy. Experiments demonstrate that Llamdex improves domain-specific accuracy by up to 26% over state-of-the-art private data synthesis methods under identical privacy constraints and, by obviating the need for users to provide domain context within queries, maintains inference efficiency comparable to the original LLM service.