Low Rank Comes with Low Security: Gradient Assembly Poisoning Attacks against Distributed LoRA-based LLM Systems

TL;DR

This paper uncovers a new attack method called Gradient Assembly Poisoning (GAP) that exploits low-rank adaptation in federated large language model systems, leading to degraded or biased outputs without detection.

Contribution

The paper introduces GAP, a novel poisoning attack exploiting the low-rank matrix assembly process in federated LoRA systems, revealing systemic vulnerabilities and demonstrating its effectiveness across multiple models.

Findings

GAP causes up to 14.5% BLEU score reduction.

GAP increases factual and grammatical errors over 800%.

GAP remains undetected by standard anomaly detectors.

Abstract

Low-Rank Adaptation (LoRA) has become a popular solution for fine-tuning large language models (LLMs) in federated settings, dramatically reducing update costs by introducing trainable low-rank matrices. However, when integrated with frameworks like FedIT, LoRA introduces a critical vulnerability: clients submit $A$ and $B$ matrices separately, while only their product $AB$ determines the model update, yet this composite is never directly verified. We propose Gradient Assembly Poisoning (GAP), a novel attack that exploits this blind spot by crafting individually benign $A$ and $B$ matrices whose product yields malicious updates. GAP operates without access to training data or inter-client coordination and remains undetected by standard anomaly detectors. We identify four systemic vulnerabilities in LoRA-based federated systems and validate GAP across LLaMA, ChatGLM, and GPT-2. GAP consistently induces degraded or biased outputs while preserving surface fluency, reducing BLEU by up to 14.5\%, increasing factual and grammatical errors by over 800\%, and maintaining 92.6\% long-form response length. These results reveal a new class of stealthy, persistent threats in distributed LoRA fine-tuning.