FedFN: Feature Normalization for Alleviating Data Heterogeneity Problem in Federated Learning

TL;DR

This paper introduces FedFN, a simple feature normalization technique for federated learning that effectively mitigates data heterogeneity issues, improving model performance especially with complex models like ResNet18 and foundation models.

Contribution

FedFN is a novel feature normalization method that addresses data heterogeneity in federated learning, demonstrating superior performance over existing approaches.

Findings

FedFN outperforms baseline methods in heterogeneous data scenarios.

FedFN improves feature representation stability across local and global models.

FedFN is effective with pretrained and foundation models.

Abstract

Federated Learning (FL) is a collaborative method for training models while preserving data privacy in decentralized settings. However, FL encounters challenges related to data heterogeneity, which can result in performance degradation. In our study, we observe that as data heterogeneity increases, feature representation in the FedAVG model deteriorates more significantly compared to classifier weight. Additionally, we observe that as data heterogeneity increases, the gap between higher feature norms for observed classes, obtained from local models, and feature norms of unobserved classes widens, in contrast to the behavior of classifier weight norms. This widening gap extends to encompass the feature norm disparities between local and the global models. To address these issues, we introduce Federated Averaging with Feature Normalization Update (FedFN), a straightforward learning method. We demonstrate the superior performance of FedFN through extensive experiments, even when applied to pretrained ResNet18. Subsequently, we confirm the applicability of FedFN to foundation models.