Enabling Quartile-based Estimated-Mean Gradient Aggregation As Baseline for Federated Image Classifications

TL;DR

This paper introduces Estimated Mean Aggregation (EMA), a novel baseline method for federated learning that improves security, handles data heterogeneity, and enhances model performance across diverse client datasets.

Contribution

The paper proposes EMA as a new baseline for federated learning aggregation, addressing security and data heterogeneity challenges with superior experimental results.

Findings

EMA outperforms alternative methods in accuracy and AUC

EMA effectively handles malicious outliers and data heterogeneity

EMA provides a robust baseline for evaluating federated learning aggregation techniques

Abstract

Federated Learning (FL) has revolutionized how we train deep neural networks by enabling decentralized collaboration while safeguarding sensitive data and improving model performance. However, FL faces two crucial challenges: the diverse nature of data held by individual clients and the vulnerability of the FL system to security breaches. This paper introduces an innovative solution named Estimated Mean Aggregation (EMA) that not only addresses these challenges but also provides a fundamental reference point as a $\mathsf{baseline}$ for advanced aggregation techniques in FL systems. EMA's significance lies in its dual role: enhancing model security by effectively handling malicious outliers through trimmed means and uncovering data heterogeneity to ensure that trained models are adaptable across various client datasets. Through a wealth of experiments, EMA consistently demonstrates high accuracy and area under the curve (AUC) compared to alternative methods, establishing itself as a robust baseline for evaluating the effectiveness and security of FL aggregation methods. EMA's contributions thus offer a crucial step forward in advancing the efficiency, security, and versatility of decentralized deep learning in the context of FL.