Stealing User Prompts from Mixture of Experts
Itay Yona, Ilia Shumailov, Jamie Hayes, Nicholas Carlini

TL;DR
This paper demonstrates a novel attack exploiting architectural flaws in Mixture-of-Experts models, enabling full prompt disclosure through carefully arranged queries, revealing a new class of vulnerabilities in language models.
Contribution
It introduces the first attack exploiting architectural flaws in MoE models to extract user prompts, highlighting a new security vulnerability.
Findings
Successfully extracted entire prompts with $O({VM}^2)$ queries
Achieved prompt extraction with an average of 100 queries per token
Demonstrated attack effectiveness on a two-layer Mixtral model
Abstract
Mixture-of-Experts (MoE) models improve the efficiency and scalability of dense language models by routing each token to a small number of experts in each layer. In this paper, we show how an adversary that can arrange for their queries to appear in the same batch of examples as a victim's queries can exploit Expert-Choice-Routing to fully disclose a victim's prompt. We successfully demonstrate the effectiveness of this attack on a two-layer Mixtral model, exploiting the tie-handling behavior of the torch.topk CUDA implementation. Our results show that we can extract the entire prompt using queries (with vocabulary size and prompt length ) or 100 queries on average per token in the setting we consider. This is the first attack to exploit architectural flaws for the purpose of extracting user prompts, introducing a new class of LLM vulnerabilities.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsData Mining Algorithms and Applications
