Equitable Federated Learning with Activation Clustering

TL;DR

This paper introduces an equitable federated learning framework that clusters clients based on activation vectors to reduce bias and ensure fairer model training across diverse groups.

Contribution

It proposes a novel clustering method using activation vectors and a client weighting mechanism to promote fairness and mitigate bias in federated learning.

Findings

Reduces bias among client clusters

Achieves $O(1/\sqrt{K})$ convergence rate

Demonstrates improved fairness over baselines

Abstract

Federated learning is a prominent distributed learning paradigm that incorporates collaboration among diverse clients, promotes data locality, and thus ensures privacy. These clients have their own technological, cultural, and other biases in the process of data generation. However, the present standard often ignores this bias/heterogeneity, perpetuating bias against certain groups rather than mitigating it. In response to this concern, we propose an equitable clustering-based framework where the clients are categorized/clustered based on how similar they are to each other. We propose a unique way to construct the similarity matrix that uses activation vectors. Furthermore, we propose a client weighing mechanism to ensure that each cluster receives equal importance and establish $O(1/\sqrt{K})$ rate of convergence to reach an $\epsilon-$stationary solution. We assess the effectiveness of our proposed strategy against common baselines, demonstrating its efficacy in terms of reducing the bias existing amongst various client clusters and consequently ameliorating algorithmic bias against specific groups.