Privacy-Preserving In-Context Learning for Large Language Models

TL;DR

This paper introduces Differentially Private In-context Learning (DP-ICL), a method to protect sensitive information in large language models' responses while maintaining high accuracy across classification and generation tasks.

Contribution

The paper proposes a novel DP-ICL paradigm that privatizes in-context learning responses using ensemble-based noisy consensus, applicable to various NLP tasks.

Findings

DP-ICL achieves a strong utility-privacy tradeoff.

Effective privatization for text classification and language generation.

Empirical results on multiple benchmarks demonstrate its effectiveness.

Abstract

In-context learning (ICL) is an important capability of Large Language Models (LLMs), enabling these models to dynamically adapt based on specific, in-context exemplars, thereby improving accuracy and relevance. However, LLM's responses may leak the sensitive private information contained in in-context exemplars. To address this challenge, we propose Differentially Private In-context Learning (DP-ICL), a general paradigm for privatizing ICL tasks. The key idea for DP-ICL paradigm is generating differentially private responses through a noisy consensus among an ensemble of LLM's responses based on disjoint exemplar sets. Based on the general paradigm of DP-ICL, we instantiate several techniques showing how to privatize ICL for text classification and language generation. We evaluate DP-ICL on four text classification benchmarks and two language generation tasks, and our empirical results show that DP-ICL achieves a strong utility-privacy tradeoff.