FedSDWC: Federated Synergistic Dual-Representation Weak Causal Learning for OOD

TL;DR

FedSDWC introduces a causal inference approach in federated learning that effectively captures invariant features and improves out-of-distribution generalization under data heterogeneity.

Contribution

It proposes a novel federated causal learning method, FedSDWC, integrating invariant and variant features for better OOD detection and generalization.

Findings

Outperforms baseline methods by 3.04% on CIFAR-10

Achieves 8.11% higher accuracy on CIFAR-100

Provides theoretical generalization error bounds

Abstract

Amid growing demands for data privacy and advances in computational infrastructure, federated learning (FL) has emerged as a prominent distributed learning paradigm. Nevertheless, differences in data distribution (such as covariate and semantic shifts) severely affect its reliability in real-world deployments. To address this issue, we propose FedSDWC, a causal inference method that integrates both invariant and variant features. FedSDWC infers causal semantic representations by modeling the weak causal influence between invariant and variant features, effectively overcoming the limitations of existing invariant learning methods in accurately capturing invariant features and directly constructing causal representations. This approach significantly enhances FL's ability to generalize and detect OOD data. Theoretically, we derive FedSDWC's generalization error bound under specific conditions and, for the first time, establish its relationship with client prior distributions. Moreover, extensive experiments conducted on multiple benchmark datasets validate the superior performance of FedSDWC in handling covariate and semantic shifts. For example, FedSDWC outperforms FedICON, the next best baseline, by an average of 3.04% on CIFAR-10 and 8.11% on CIFAR-100.