FedProphet: Memory-Efficient Federated Adversarial Training via Robust and Consistent Cascade Learning

TL;DR

FedProphet introduces a memory-efficient federated adversarial training framework that enhances robustness and consistency on resource-constrained devices, achieving significant speedups and memory savings without sacrificing accuracy.

Contribution

The paper presents FedProphet, a novel framework combining memory efficiency, adversarial robustness, and model consistency through cascade learning and server-side coordination techniques.

Findings

Achieves 80% memory reduction in training.

Maintains high accuracy and robustness comparable to full-memory methods.

Speeds up training by up to 10.8 times.

Abstract

Federated Adversarial Training (FAT) can supplement robustness against adversarial examples to Federated Learning (FL), promoting a meaningful step toward trustworthy AI. However, FAT requires large models to preserve high accuracy while achieving strong robustness, incurring high memory-swapping latency when training on memory-constrained edge devices. Existing memory-efficient FL methods suffer from poor accuracy and weak robustness due to inconsistent local and global models. In this paper, we propose FedProphet, a novel FAT framework that can achieve memory efficiency, robustness, and consistency simultaneously. FedProphget reduces the memory requirement in local training while guaranteeing adversarial robustness by adversarial cascade learning with strong convexity regularization, and we show that the strong robustness also implies low inconsistency in FedProphet. We also develop a training coordinator on the server of FL, with Adaptive Perturbation Adjustment for utility-robustness balance and Differentiated Module Assignment for objective inconsistency mitigation. FedPeophet significantly outperforms other baselines under different experimental settings, maintaining the accuracy and robustness of end-to-end FAT with 80% memory reduction and up to 10.8x speedup in training time.