PrivAct: Internalizing Contextual Privacy Preservation via Multi-Agent Preference Training

TL;DR

PrivAct introduces a multi-agent learning framework that internalizes privacy preferences into LLMs, improving privacy preservation without sacrificing helpfulness across various benchmarks.

Contribution

It is the first approach to embed privacy preferences directly into LLMs for context-aware privacy preservation in multi-agent systems.

Findings

Reduces privacy leakage rates by up to 12.32%.

Maintains comparable helpfulness to baseline models.

Demonstrates robustness and zero-shot generalization across diverse settings.

Abstract

Large language model (LLM) agents are increasingly deployed in personalized tasks involving sensitive, context-dependent information, where privacy violations may arise in agents' action due to the implicitness of contextual privacy. Existing approaches rely on external, inference-time interventions which are brittle, scenario-specific, and may expand the privacy attack surface. We propose PrivAct, a contextual privacy-aware multi-agent learning framework that internalizes contextual privacy preservation directly into models' generation behavior for privacy-compliant agentic actions. By embedding privacy preferences into each agent, PrivAct enhances system-wide contextual integrity while achieving a more favorable privacy-helpfulness tradeoff. Experiments across multiple LLM backbones and benchmarks demonstrate consistent improvements in contextual privacy preservation, reducing leakage rates by up to 12.32% while maintaining comparable helpfulness, as well as zero-shot generalization and robustness across diverse multi-agent topologies. Code is available at https://github.com/chengyh23/PrivAct.