FedCFA: Alleviating Simpson's Paradox in Model Aggregation with Counterfactual Federated Learning

TL;DR

FedCFA introduces a counterfactual learning framework in federated learning to address Simpson's Paradox caused by data heterogeneity, improving global model accuracy and efficiency.

Contribution

The paper proposes FedCFA, a novel federated learning approach using counterfactual samples and factor decorrelation to mitigate Simpson's Paradox effects in non-IID data.

Findings

Outperforms existing FL methods in accuracy and efficiency

Effectively mitigates Simpson's Paradox in heterogeneous data scenarios

Achieves superior results on six benchmark datasets

Abstract

Federated learning (FL) is a promising technology for data privacy and distributed optimization, but it suffers from data imbalance and heterogeneity among clients. Existing FL methods try to solve the problems by aligning client with server model or by correcting client model with control variables. These methods excel on IID and general Non-IID data but perform mediocrely in Simpson's Paradox scenarios. Simpson's Paradox refers to the phenomenon that the trend observed on the global dataset disappears or reverses on a subset, which may lead to the fact that global model obtained through aggregation in FL does not accurately reflect the distribution of global data. Thus, we propose FedCFA, a novel FL framework employing counterfactual learning to generate counterfactual samples by replacing local data critical factors with global average data, aligning local data distributions with the global and mitigating Simpson's Paradox effects. In addition, to improve the quality of counterfactual samples, we introduce factor decorrelation (FDC) loss to reduce the correlation among features and thus improve the independence of extracted factors. We conduct extensive experiments on six datasets and verify that our method outperforms other FL methods in terms of efficiency and global model accuracy under limited communication rounds.