FedGA: Federated Learning with Gradient Alignment for Error Asymmetry Mitigation

TL;DR

FedGA introduces a gradient alignment approach in federated learning to mitigate error asymmetry bias, improving model convergence and accuracy over traditional methods.

Contribution

The paper proposes FedGA, a novel federated learning method that uses gradient alignment via label calibration to address bias caused by error asymmetry, outperforming existing algorithms.

Findings

FedGA reduces error asymmetry more effectively than FedAvg.

It achieves higher F1 scores and accuracy margins on benchmark datasets.

Performance improves with increased Dirichlet sampling factor .

Abstract

Federated learning (FL) triggers intra-client and inter-client class imbalance, with the latter compared to the former leading to biased client updates and thus deteriorating the distributed models. Such a bias is exacerbated during the server aggregation phase and has yet to be effectively addressed by conventional re-balancing methods. To this end, different from the off-the-shelf label or loss-based approaches, we propose a gradient alignment (GA)-informed FL method, dubbed as FedGA, where the importance of error asymmetry (EA) in bias is observed and its linkage to the gradient of the loss to raw logits is explored. Concretely, GA, implemented by label calibration during the model backpropagation process, prevents catastrophic forgetting of rate and missing classes, hence boosting model convergence and accuracy. Experimental results on five benchmark datasets demonstrate that GA outperforms the pioneering counterpart FedAvg and its four variants in minimizing EA and updating bias, and accordingly yielding higher F1 score and accuracy margins when the Dirichlet distribution sampling factor $\alpha$ increases. The code and more details are available at \url{https://anonymous.4open.science/r/FedGA-B052/README.md}.