Information-Theoretic Privacy Control for Sequential Multi-Agent LLM Systems

TL;DR

This paper analyzes privacy risks in sequential multi-agent LLM systems, formalizes information leakage, and proposes a privacy-regularized training method to control system-level privacy while maintaining utility.

Contribution

It introduces a formal mutual information-based framework for privacy leakage in sequential LLM agents and develops a training approach to mitigate systemic privacy risks.

Findings

Leakage amplifies across agents in sequential pipelines.

The proposed method achieves stable privacy-utility trade-offs.

Privacy cannot be ensured by local constraints alone.

Abstract

Sequential multi-agent large language model (LLM) systems are increasingly deployed in sensitive domains such as healthcare, finance, and enterprise decision-making, where multiple specialized agents collaboratively process a single user request. Although individual agents may satisfy local privacy constraints, sensitive information can still be inferred through sequential composition and intermediate representations. In this work, we study \emph{compositional privacy leakage} in sequential LLM agent pipelines. We formalize leakage using mutual information and derive a theoretical bound that characterizes how locally introduced leakage can amplify across agents under sequential execution. Motivated by this analysis, we propose a privacy-regularized training framework that directly constrains information flow between agent outputs and agent-local sensitive variables. We evaluate our approach across sequential agent pipelines of varying depth on three benchmark datasets, demonstrating stable optimization dynamics and consistent, interpretable privacy-utility trade-offs. Our results show that privacy in agentic LLM systems cannot be guaranteed by local constraints alone and must instead be treated as a system-level property during both training and deployment.