ScaleOT: Privacy-utility-scalable Offsite-tuning with Dynamic LayerReplace and Selective Rank Compression

TL;DR

ScaleOT is a scalable offsite-tuning framework that uses reinforcement learning and layer compression to balance privacy and utility in large language models, outperforming existing methods in privacy protection with minimal utility loss.

Contribution

It introduces a novel layerwise compression and harmonizer replacement approach, enabling privacy-preserving offsite tuning with improved scalability and performance.

Findings

Achieves near lossless tuning performance compared to full fine-tuning.

Provides enhanced privacy protection through rank reduction and layer importance-based compression.

Maintains utility with negligible impact despite significant compression.

Abstract

Offsite-tuning is a privacy-preserving method for tuning large language models (LLMs) by sharing a lossy compressed emulator from the LLM owners with data owners for downstream task tuning. This approach protects the privacy of both the model and data owners. However, current offsite tuning methods often suffer from adaptation degradation, high computational costs, and limited protection strength due to uniformly dropping LLM layers or relying on expensive knowledge distillation. To address these issues, we propose ScaleOT, a novel privacy-utility-scalable offsite-tuning framework that effectively balances privacy and utility. ScaleOT introduces a novel layerwise lossy compression algorithm that uses reinforcement learning to obtain the importance of each layer. It employs lightweight networks, termed harmonizers, to replace the raw LLM layers. By combining important original LLM layers and harmonizers in different ratios, ScaleOT generates emulators tailored for optimal performance with various model scales for enhanced privacy protection. Additionally, we present a rank reduction method to further compress the original LLM layers, significantly enhancing privacy with negligible impact on utility. Comprehensive experiments show that ScaleOT can achieve nearly lossless offsite tuning performance compared with full fine-tuning while obtaining better model privacy.