Federated Unsupervised Domain Generalization using Global and Local Alignment of Gradients

TL;DR

This paper introduces FedGaLA, a novel federated learning method that aligns gradients at client and server levels to improve model generalization across unseen domains in an unsupervised setting.

Contribution

The paper presents a new gradient alignment approach for federated unsupervised domain generalization, connecting domain shift with gradient alignment and demonstrating its effectiveness.

Findings

FedGaLA outperforms baseline methods on four multi-domain datasets.

Gradient alignment at client and server levels improves domain-invariant feature learning.

Ablation studies confirm the importance of each component in FedGaLA.

Abstract

We address the problem of federated domain generalization in an unsupervised setting for the first time. We first theoretically establish a connection between domain shift and alignment of gradients in unsupervised federated learning and show that aligning the gradients at both client and server levels can facilitate the generalization of the model to new (target) domains. Building on this insight, we propose a novel method named FedGaLA, which performs gradient alignment at the client level to encourage clients to learn domain-invariant features, as well as global gradient alignment at the server to obtain a more generalized aggregated model. To empirically evaluate our method, we perform various experiments on four commonly used multi-domain datasets, PACS, OfficeHome, DomainNet, and TerraInc. The results demonstrate the effectiveness of our method which outperforms comparable baselines. Ablation and sensitivity studies demonstrate the impact of different components and parameters in our approach. The source code is available at: https://github.com/MahdiyarMM/FedGaLA.