Lorica: A Synergistic Fine-Tuning Framework for Advancing Personalized Adversarial Robustness

TL;DR

Lorica is a personalized federated adversarial training framework that enhances robustness and accuracy on edge devices while significantly reducing communication costs through a two-phase process involving local fine-tuning and strategic model selection.

Contribution

It introduces a novel two-phase personalized adversarial training method using LoRA-FA and forward-gating, improving robustness, accuracy, and communication efficiency in federated learning.

Findings

Achieves up to 68x communication efficiency improvements.

Improves adversarial robustness by up to 29.9%.

Enhances benign accuracy by up to 52.2%.

Abstract

The growing use of large pre-trained models in edge computing has made model inference on mobile clients both feasible and popular. Yet these devices remain vulnerable to adversarial attacks, threatening model robustness and security. Federated adversarial training (FAT) offers a promising solution by enhancing robustness while preserving client privacy. However, FAT often yields a generalized global model that struggles with heterogeneous client data, leading to limited personalization and significant communication overhead. In this paper, we propose \textit{Lorica}, a personalized synergistic adversarial training framework that delivers customized defense models through a two-phase process. In Phase 1, \textit{Lorica} applies LoRA-FA for local adversarial fine-tuning, enabling personalized robustness while reducing communication by uploading only LoRA-FA parameters. In Phase 2, a forward-gating selection strategy improves benign accuracy, further refining the personalized model. This yields tailored defense models that effectively balance robustness and accuracy. Extensive experiments on benchmark datasets demonstrate that \textit{Lorica} can achieve up to 68$\times$ improvements in communication efficiency compared to state-of-the-art algorithms, while achieving up to 29.9\% and 52.2\% enhancements in adversarial robustness and benign accuracy, respectively.