Provable Differentially Private Computation of the Cross-Attention Mechanism

TL;DR

This paper introduces a novel differentially private data structure for cross-attention mechanisms in AI models, providing provable privacy guarantees while maintaining efficiency and robustness against adaptive queries.

Contribution

It presents the first provably differentially private cross-attention mechanism with theoretical guarantees and practical efficiency for large generative models.

Findings

Achieves $ ilde{O}(ndr^2)$ space and initialization complexity.

Satisfies $( ext{epsilon}, ext{delta})$-DP with bounded error.

Maintains robustness against adaptive, adversarial queries.

Abstract

Cross-attention has emerged as a cornerstone module in modern artificial intelligence, underpinning critical applications such as retrieval-augmented generation (RAG), system prompting, and guided stable diffusion. However, this is a rising concern about securing the privacy of cross-attention, as the underlying key and value matrices frequently encode sensitive data or private user information. In this work, we introduce a novel data structure designed to enforce differential privacy (DP) for cross-attention mechanisms, accompanied by provable theoretical guarantees. Specifically, letting $n$ denote the input sequence length, $d$ the feature dimension, $R$ the maximum magnitude of query and key matrices, $R_w$ the maximum magnitude of the value matrix, and $r, s, \epsilon_s$ the parameters for polynomial kernel methods, our proposed structure achieves $\widetilde{O}(ndr^2)$ space and initialization complexity, with a query time of $\widetilde{O}(d r^2)$ per token. Moreover, we demonstrate that our mechanism satisfies $(\epsilon, \delta)$-DP, incurring an additive error of $\widetilde{O}((1-\epsilon_s)^{-1} n^{-1} \epsilon^{-1} R^{2s} R_w r^2)$ and a relative error of $2\epsilon_s/(1-\epsilon_s)$ with respect to the ground truth. Crucially, our framework maintains robustness against adaptive queries, ensuring security even in adversarial settings. To the best of our knowledge, this constitutes the first approach providing provable differential privacy for cross-attention, establishing a foundation for future privacy-preserving algorithms in large generative models (LGMs).