Graph Representation Learning Augmented Model Manipulation on Federated Fine-Tuning of LLMs

TL;DR

This paper introduces AugMP, a graph-based manipulation strategy for federated fine-tuning of LLMs, significantly degrading model performance while evading defenses.

Contribution

It proposes a novel graph representation learning framework and an iterative manipulation algorithm to enhance stealthy adversarial attacks on federated LLM fine-tuning.

Findings

AugMP reduces global LLM accuracy by up to 26%.

It degrades local LLM agents' accuracy by up to 22%.

AugMP effectively evades conventional defense methods.

Abstract

Federated fine-tuning (FFT) has emerged as a privacy-preserving paradigm for collaboratively adapting large language models (LLMs). Built upon federated learning, FFT enables distributed agents to jointly refine a shared pretrained LLM by aggregating local LLM updates without sharing local raw data. However, FFT-based LLMs remain vulnerable to model manipulation threats, in which adversarial participants upload manipulated LLM updates that corrupt the aggregation process and degrade the performance of the global LLM. In this paper, we propose an Augmented Model maniPulation (AugMP) strategy against FFT-based LLMs. Specifically, we design a novel graph representation learning framework that captures feature correlations among benign LLM updates to guide the generation of malicious updates. To enhance manipulation effectiveness and stealthiness, we develop an iterative manipulation algorithm based on an augmented Lagrangian dual formulation. Through this formulation, malicious updates are optimized to embed adversarial objectives while preserving benign-like parameter characteristics. Experimental results across multiple LLM backbones demonstrate that the AugMP strategy achieves the strongest manipulation performance among all competing baselines, reducing the global LLM accuracy by up to 26% and degrading the average accuracy of local LLM agents by up to 22%. Meanwhile, AugMP maintains high statistical and geometric consistency with benign updates, enabling it to evade conventional distance- and similarity-based defense methods.