Federated Learning with Relative Fairness

TL;DR

This paper introduces a federated learning framework focused on reducing relative unfairness among clients by using a minimax approach and a novel fairness index, with theoretical guarantees and empirical validation.

Contribution

It presents a new framework for relative fairness in federated learning, extending DRO methods and proposing an efficient algorithm with theoretical and empirical support.

Findings

Framework effectively reduces client performance disparities.

Theoretical guarantees ensure consistent unfairness reduction.

Empirical results confirm maintained performance and fairness improvements.

Abstract

This paper proposes a federated learning framework designed to achieve \textit{relative fairness} for clients. Traditional federated learning frameworks typically ensure absolute fairness by guaranteeing minimum performance across all client subgroups. However, this approach overlooks disparities in model performance between subgroups. The proposed framework uses a minimax problem approach to minimize relative unfairness, extending previous methods in distributionally robust optimization (DRO). A novel fairness index, based on the ratio between large and small losses among clients, is introduced, allowing the framework to assess and improve the relative fairness of trained models. Theoretical guarantees demonstrate that the framework consistently reduces unfairness. We also develop an algorithm, named \textsc{Scaff-PD-IA}, which balances communication and computational efficiency while maintaining minimax-optimal convergence rates. Empirical evaluations on real-world datasets confirm its effectiveness in maintaining model performance while reducing disparity.