Feature Matching Data Synthesis for Non-IID Federated Learning

TL;DR

This paper introduces a feature matching data synthesis method to generate synthetic data that addresses non-IID data challenges in federated learning, improving model accuracy and privacy.

Contribution

It proposes a novel HFMDS approach for data augmentation in federated learning, enhancing generalization and privacy while effectively tackling data heterogeneity.

Findings

Outperforms baseline methods in accuracy on benchmark datasets

Enhances privacy preservation through feature augmentation

Reduces computational cost compared to existing approaches

Abstract

Federated learning (FL) has emerged as a privacy-preserving paradigm that trains neural networks on edge devices without collecting data at a central server. However, FL encounters an inherent challenge in dealing with non-independent and identically distributed (non-IID) data among devices. To address this challenge, this paper proposes a hard feature matching data synthesis (HFMDS) method to share auxiliary data besides local models. Specifically, synthetic data are generated by learning the essential class-relevant features of real samples and discarding the redundant features, which helps to effectively tackle the non-IID issue. For better privacy preservation, we propose a hard feature augmentation method to transfer real features towards the decision boundary, with which the synthetic data not only improve the model generalization but also erase the information of real features. By integrating the proposed HFMDS method with FL, we present a novel FL framework with data augmentation to relieve data heterogeneity. The theoretical analysis highlights the effectiveness of our proposed data synthesis method in solving the non-IID challenge. Simulation results further demonstrate that our proposed HFMDS-FL algorithm outperforms the baselines in terms of accuracy, privacy preservation, and computational cost on various benchmark datasets.