Anti-Matthew FL: Bridging the Performance Gap in Federated Learning to Counteract the Matthew Effect

TL;DR

This paper introduces anti-Matthew federated learning, a method designed to ensure fairness among clients by balancing performance disparities caused by the Matthew effect, using a multi-objective optimization approach.

Contribution

It proposes a novel anti-Matthew fairness framework in federated learning, formalizes it as a multi-constrained multi-objectives optimization problem, and develops a three-stage multi-gradient descent algorithm.

Findings

Outperforms state-of-the-art FL algorithms in accuracy and fairness.

Effectively reduces performance gaps among clients.

Ensures high-quality global model with fair client performance.

Abstract

Federated learning (FL) stands as a paradigmatic approach that facilitates model training across heterogeneous and diverse datasets originating from various data providers. However, conventional FLs fall short of achieving consistent performance, potentially leading to performance degradation for clients who are disadvantaged in data resources. Influenced by the Matthew effect, deploying a performance-imbalanced global model in applications further impedes the generation of high-quality data from disadvantaged clients, exacerbating the disparities in data resources among clients. In this work, we propose anti-Matthew fairness for the global model at the client level, requiring equal accuracy and equal decision bias across clients. To balance the trade-off between achieving anti-Matthew fairness and performance optimality, we formalize the anti-Matthew effect federated learning (anti-Matthew FL) as a multi-constrained multi-objectives optimization (MCMOO) problem and propose a three-stage multi-gradient descent algorithm to obtain the Pareto optimality. We theoretically analyze the convergence and time complexity of our proposed algorithms. Additionally, through extensive experimentation, we demonstrate that our proposed anti-Matthew FL outperforms other state-of-the-art FL algorithms in achieving a high-performance global model while effectively bridging performance gaps among clients. We hope this work provides valuable insights into the manifestation of the Matthew effect in FL and other decentralized learning scenarios and can contribute to designing fairer learning mechanisms, ultimately fostering societal welfare.