Robust Federated Learning by Mixture of Experts

TL;DR

This paper introduces a robust federated learning method using a mixture of experts to mitigate the impact of poisoned, corrupted, or outdated local models, improving accuracy in non-IID data scenarios.

Contribution

It proposes a novel MoE-based aggregation technique for federated learning that enhances robustness against malicious or faulty local models.

Findings

MoE-FL outperforms traditional methods under high poisoning rates.

The approach effectively identifies and reduces outlier influence.

Robustness is improved without significant loss of accuracy.

Abstract

We present a novel weighted average model based on the mixture of experts (MoE) concept to provide robustness in Federated learning (FL) against the poisoned/corrupted/outdated local models. These threats along with the non-IID nature of data sets can considerably diminish the accuracy of the FL model. Our proposed MoE-FL setup relies on the trust between users and the server where the users share a portion of their public data sets with the server. The server applies a robust aggregation method by solving the optimization problem or the Softmax method to highlight the outlier cases and to reduce their adverse effect on the FL process. Our experiments illustrate that MoE-FL outperforms the performance of the traditional aggregation approach for high rate of poisoned data from attackers.