Completely Heterogeneous Federated Learning

TL;DR

This paper introduces a novel federated learning framework that enables multiple clients with completely heterogeneous data, models, and label distributions to collaboratively learn without exposing private information, using parameter decoupling and data-free knowledge distillation.

Contribution

It proposes the first framework for federated learning under fully heterogeneous conditions, addressing privacy and heterogeneity simultaneously.

Findings

Achieves high performance in completely heterogeneous scenarios.

Outperforms existing methods that fail under these conditions.

Maintains privacy by not exposing feature space, model architecture, or label distribution.

Abstract

Federated learning (FL) faces three major difficulties: cross-domain, heterogeneous models, and non-i.i.d. labels scenarios. Existing FL methods fail to handle the above three constraints at the same time, and the level of privacy protection needs to be lowered (e.g., the model architecture and data category distribution can be shared). In this work, we propose the challenging "completely heterogeneous" scenario in FL, which refers to that each client will not expose any private information including feature space, model architecture, and label distribution. We then devise an FL framework based on parameter decoupling and data-free knowledge distillation to solve the problem. Experiments show that our proposed method achieves high performance in completely heterogeneous scenarios where other approaches fail.