Certified Robustness in Federated Learning

TL;DR

This paper investigates how federated learning techniques, especially federated averaging and personalization, impact the certified robustness of models against adversarial input transformations using randomized smoothing.

Contribution

It demonstrates that federated averaging enhances both accuracy and robustness, and analyzes how personalization affects model robustness and training efficiency.

Findings

Federated averaging improves model robustness over local training.

Personalization increases robustness and speeds up training.

Robustness of local models decreases as they diverge from the global model.

Abstract

Federated learning has recently gained significant attention and popularity due to its effectiveness in training machine learning models on distributed data privately. However, as in the single-node supervised learning setup, models trained in federated learning suffer from vulnerability to imperceptible input transformations known as adversarial attacks, questioning their deployment in security-related applications. In this work, we study the interplay between federated training, personalization, and certified robustness. In particular, we deploy randomized smoothing, a widely-used and scalable certification method, to certify deep networks trained on a federated setup against input perturbations and transformations. We find that the simple federated averaging technique is effective in building not only more accurate, but also more certifiably-robust models, compared to training solely on local data. We further analyze personalization, a popular technique in federated training that increases the model's bias towards local data, on robustness. We show several advantages of personalization over both~(that is, only training on local data and federated training) in building more robust models with faster training. Finally, we explore the robustness of mixtures of global and local~(i.e. personalized) models, and find that the robustness of local models degrades as they diverge from the global model