UniVarFL: Uniformity and Variance Regularized Federated Learning for Heterogeneous Data

TL;DR

UniVarFL introduces a federated learning framework that uses regularization techniques at the client level to address data heterogeneity, improving model accuracy without relying on global models.

Contribution

It proposes a novel FL method with classifier variance and hyperspherical uniformity regularizations, eliminating the need for global model dependency and enhancing generalization.

Findings

Outperforms existing methods on benchmark datasets

Improves accuracy in non-IID data scenarios

Reduces computational costs compared to traditional approaches

Abstract

Federated Learning (FL) often suffers from severe performance degradation when faced with non-IID data, largely due to local classifier bias. Traditional remedies such as global model regularization or layer freezing either incur high computational costs or struggle to adapt to feature shifts. In this work, we propose UniVarFL, a novel FL framework that emulates IID-like training dynamics directly at the client level, eliminating the need for global model dependency. UniVarFL leverages two complementary regularization strategies during local training: Classifier Variance Regularization, which aligns class-wise probability distributions with those expected under IID conditions, effectively mitigating local classifier bias; and Hyperspherical Uniformity Regularization, which encourages a uniform distribution of feature representations across the hypersphere, thereby enhancing the model's ability to generalize under diverse data distributions. Extensive experiments on multiple benchmark datasets demonstrate that UniVarFL outperforms existing methods in accuracy, highlighting its potential as a highly scalable and efficient solution for real-world FL deployments, especially in resource-constrained settings. Code: https://github.com/sunnyinAI/UniVarFL