Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Models

TL;DR

This paper introduces a method for privately learning prompts for large language models using a flock of stochastic parrots, effectively balancing privacy with high downstream task performance.

Contribution

It proposes a novel differentially private prompt learning technique that uses ensemble voting among multiple LLM prompts to preserve privacy without sacrificing much accuracy.

Findings

Achieves 92.7% accuracy with differential privacy on SST-2.

Private prompt learning closely matches non-private performance.

Method is compatible with existing commercial LLM APIs.

Abstract

Large language models (LLMs) are excellent in-context learners. However, the sensitivity of data contained in prompts raises privacy concerns. Our work first shows that these concerns are valid: we instantiate a simple but highly effective membership inference attack against the data used to prompt LLMs. To address this vulnerability, one could forego prompting and resort to fine-tuning LLMs with known algorithms for private gradient descent. However, this comes at the expense of the practicality and efficiency offered by prompting. Therefore, we propose to privately learn to prompt. We first show that soft prompts can be obtained privately through gradient descent on downstream data. However, this is not the case for discrete prompts. Thus, we orchestrate a noisy vote among an ensemble of LLMs presented with different prompts, i.e., a flock of stochastic parrots. The vote privately transfers the flock's knowledge into a single public prompt. We show that LLMs prompted with our private algorithms closely match the non-private baselines. For example, using GPT3 as the base model, we achieve a downstream accuracy of 92.7% on the sst2 dataset with ($\epsilon=0.147, \delta=10^{-6}$)-differential privacy vs. 95.2% for the non-private baseline. Through our experiments, we also show that our prompt-based approach is easily deployed with existing commercial APIs.