Federated Adversarial Training with Transformers

TL;DR

This paper explores the application of adversarial training in federated learning for vision transformers, proposing a new aggregation method, FedWAvg, to enhance model robustness against adversarial attacks.

Contribution

It introduces FedWAvg, a novel federated aggregation technique that improves robustness of vision transformer models trained with adversarial training in non-IID settings.

Findings

FedWAvg outperforms existing aggregation methods in robust accuracy.

Adversarial training is feasible and effective for vision transformers in federated learning.

The proposed method enhances model robustness against evasion attacks.

Abstract

Federated learning (FL) has emerged to enable global model training over distributed clients' data while preserving its privacy. However, the global trained model is vulnerable to the evasion attacks especially, the adversarial examples (AEs), carefully crafted samples to yield false classification. Adversarial training (AT) is found to be the most promising approach against evasion attacks and it is widely studied for convolutional neural network (CNN). Recently, vision transformers have been found to be effective in many computer vision tasks. To the best of the authors' knowledge, there is no work that studied the feasibility of AT in a FL process for vision transformers. This paper investigates such feasibility with different federated model aggregation methods and different vision transformer models with different tokenization and classification head techniques. In order to improve the robust accuracy of the models with the not independent and identically distributed (Non-IID), we propose an extension to FedAvg aggregation method, called FedWAvg. By measuring the similarities between the last layer of the global model and the last layer of the client updates, FedWAvg calculates the weights to aggregate the local models updates. The experiments show that FedWAvg improves the robust accuracy when compared with other state-of-the-art aggregation methods.