AdaFed: Fair Federated Learning via Adaptive Common Descent Direction

TL;DR

AdaFed introduces an adaptive method for federated learning that ensures fairer model training by dynamically adjusting the update direction to benefit all clients, especially those with higher loss values.

Contribution

This work proposes AdaFed, a novel adaptive approach for fair federated learning that optimizes the update direction based on local gradients and loss functions.

Findings

AdaFed outperforms existing fair FL methods on multiple datasets.

It effectively reduces unfairness in model training across clients.

The method adapts to client-specific loss dynamics for improved fairness.

Abstract

Federated learning (FL) is a promising technology via which some edge devices/clients collaboratively train a machine learning model orchestrated by a server. Learning an unfair model is known as a critical problem in federated learning, where the trained model may unfairly advantage or disadvantage some of the devices. To tackle this problem, in this work, we propose AdaFed. The goal of AdaFed is to find an updating direction for the server along which (i) all the clients' loss functions are decreasing; and (ii) more importantly, the loss functions for the clients with larger values decrease with a higher rate. AdaFed adaptively tunes this common direction based on the values of local gradients and loss functions. We validate the effectiveness of AdaFed on a suite of federated datasets, and demonstrate that AdaFed outperforms state-of-the-art fair FL methods.