FedSM: Robust Semantics-Guided Feature Mixup for Bias Reduction in Federated Learning with Long-Tail Data

TL;DR

FedSM is a client-centric federated learning framework that uses semantics-guided feature mixup and classifier retraining to reduce bias caused by long-tail data distributions, improving accuracy and robustness.

Contribution

Introduces FedSM, a novel semantics-guided feature mixup approach with probabilistic category selection to mitigate bias in federated learning with long-tail data.

Findings

Outperforms state-of-the-art methods in accuracy on long-tail datasets.

Demonstrates high robustness to domain shift.

Requires minimal server overhead and computational resources.

Abstract

Federated Learning (FL) enables collaborative model training across decentralized clients without sharing private data. However, FL suffers from biased global models due to non-IID and long-tail data distributions. We propose \textbf{FedSM}, a novel client-centric framework that mitigates this bias through semantics-guided feature mixup and lightweight classifier retraining. FedSM uses a pretrained image-text-aligned model to compute category-level semantic relevance, guiding the category selection of local features to mix-up with global prototypes to generate class-consistent pseudo-features. These features correct classifier bias, especially when data are heavily skewed. To address the concern of potential domain shift between the pretrained model and the data, we propose probabilistic category selection, enhancing feature diversity to effectively mitigate biases. All computations are performed locally, requiring minimal server overhead. Extensive experiments on long-tail datasets with various imbalanced levels demonstrate that FedSM consistently outperforms state-of-the-art methods in accuracy, with high robustness to domain shift and computational efficiency.