AirGapAgent: Protecting Privacy-Conscious Conversational Agents

TL;DR

AirGapAgent is a privacy-preserving conversational agent that limits data access to prevent malicious context manipulation, significantly reducing information leakage in large language model interactions.

Contribution

The paper introduces AirGapAgent, a novel framework based on contextual integrity to enhance privacy in LLM-based agents by restricting unnecessary data access.

Findings

AirGapAgent achieves 97% protection against context hijacking attacks.

Traditional agents' data protection drops to 45% under attack, while AirGapAgent maintains high protection.

Extensive experiments validate the effectiveness of AirGapAgent across multiple LLM models.

Abstract

The growing use of large language model (LLM)-based conversational agents to manage sensitive user data raises significant privacy concerns. While these agents excel at understanding and acting on context, this capability can be exploited by malicious actors. We introduce a novel threat model where adversarial third-party apps manipulate the context of interaction to trick LLM-based agents into revealing private information not relevant to the task at hand. Grounded in the framework of contextual integrity, we introduce AirGapAgent, a privacy-conscious agent designed to prevent unintended data leakage by restricting the agent's access to only the data necessary for a specific task. Extensive experiments using Gemini, GPT, and Mistral models as agents validate our approach's effectiveness in mitigating this form of context hijacking while maintaining core agent functionality. For example, we show that a single-query context hijacking attack on a Gemini Ultra agent reduces its ability to protect user data from 94% to 45%, while an AirGapAgent achieves 97% protection, rendering the same attack ineffective.