FEDTAIL: Federated Long-Tailed Domain Generalization with Sharpness-Guided Gradient Matching

TL;DR

FedTAIL introduces a federated domain generalization framework that leverages sharpness-guided, gradient-aligned optimization to improve robustness and performance under long-tailed class distributions and domain shifts.

Contribution

It proposes a novel federated DG method combining sharpness-aware regularization, gradient coherence, and class-wise adaptive weighting to handle class imbalance and domain shifts.

Findings

Achieves state-of-the-art results on domain generalization benchmarks.

Effectively handles class imbalance and domain shifts in federated settings.

Demonstrates robustness and stability through sharpness-aware optimization techniques.

Abstract

Domain Generalization (DG) seeks to train models that perform reliably on unseen target domains without access to target data during training. While recent progress in smoothing the loss landscape has improved generalization, existing methods often falter under long-tailed class distributions and conflicting optimization objectives. We introduce FedTAIL, a federated domain generalization framework that explicitly addresses these challenges through sharpness-guided, gradient-aligned optimization. Our method incorporates a gradient coherence regularizer to mitigate conflicts between classification and adversarial objectives, leading to more stable convergence. To combat class imbalance, we perform class-wise sharpness minimization and propose a curvature-aware dynamic weighting scheme that adaptively emphasizes underrepresented tail classes. Furthermore, we enhance conditional distribution alignment by integrating sharpness-aware perturbations into entropy regularization, improving robustness under domain shift. FedTAIL unifies optimization harmonization, class-aware regularization, and conditional alignment into a scalable, federated-compatible framework. Extensive evaluations across standard domain generalization benchmarks demonstrate that FedTAIL achieves state-of-the-art performance, particularly in the presence of domain shifts and label imbalance, validating its effectiveness in both centralized and federated settings. Code: https://github.com/sunnyinAI/FedTail