FedLF: Adaptive Logit Adjustment and Feature Optimization in Federated Long-Tailed Learning

TL;DR

FedLF is a novel federated learning method that addresses long-tailed data distribution and class imbalance by introducing adaptive logit adjustment, class-centered optimization, and feature decorrelation, improving model performance on heterogeneous datasets.

Contribution

The paper proposes FedLF, a new federated learning approach that effectively handles long-tailed data and class imbalance through three innovative training modifications.

Findings

Outperforms existing methods on CIFAR-10-LT and CIFAR-100-LT datasets.

Effectively mitigates performance degradation caused by data heterogeneity.

Demonstrates significant improvements in class-wise accuracy, especially for tail classes.

Abstract

Federated learning offers a paradigm to the challenge of preserving privacy in distributed machine learning. However, datasets distributed across each client in the real world are inevitably heterogeneous, and if the datasets can be globally aggregated, they tend to be long-tailed distributed, which greatly affects the performance of the model. The traditional approach to federated learning primarily addresses the heterogeneity of data among clients, yet it fails to address the phenomenon of class-wise bias in global long-tailed data. This results in the trained model focusing on the head classes while neglecting the equally important tail classes. Consequently, it is essential to develop a methodology that considers classes holistically. To address the above problems, we propose a new method FedLF, which introduces three modifications in the local training phase: adaptive logit adjustment, continuous class centred optimization, and feature decorrelation. We compare seven state-of-the-art methods with varying degrees of data heterogeneity and long-tailed distribution. Extensive experiments on benchmark datasets CIFAR-10-LT and CIFAR-100-LT demonstrate that our approach effectively mitigates the problem of model performance degradation due to data heterogeneity and long-tailed distribution. our code is available at https://github.com/18sym/FedLF.