Fairness Regularization in Federated Learning

TL;DR

This paper investigates fairness regularization techniques in federated learning, proposing new methods and analyzing their effectiveness in improving fairness and model performance across diverse client data.

Contribution

It introduces and evaluates new fairness regularization approaches, clarifies relationships between existing methods, and demonstrates improved fairness and performance in heterogeneous settings.

Findings

FairGrad and FairGrad* improve fairness and accuracy

Theoretical links between fairness methods are established

Regularization enhances performance in diverse data environments

Abstract

Federated Learning (FL) has emerged as a vital paradigm in modern machine learning that enables collaborative training across decentralized data sources without exchanging raw data. This approach not only addresses privacy concerns but also allows access to overall substantially larger and potentially more diverse datasets, without the need for centralized storage or hardware resources. However, heterogeneity in client data may cause certain clients to have disproportionate impacts on the global model, leading to disparities in the clients' performances. Fairness, therefore, becomes a crucial concern in FL and can be addressed in various ways. However, the effectiveness of existing fairness-aware methods, particularly in heterogeneous data settings, remains unclear, and the relationships between different approaches are not well understood. In this work, we focus on performance equitable fairness, which aims to minimize differences in performance across clients. We restrict our study to fairness-aware methods that explicitly regularize client losses, evaluating both existing and newly proposed approaches. We identify and theoretically explain connections between the investigated fairness methods, and empirically show that FairGrad (approximate) and FairGrad* (exact) (two variants of a gradient variance regularization method introduced here for performance equitable fairness) improve both fairness and overall model performance in heterogeneous data settings.