Continual Horizontal Federated Learning for Heterogeneous Data

TL;DR

This paper introduces CHFL, a novel continual horizontal federated learning method that effectively utilizes both common and client-specific features, significantly enhancing model performance on heterogeneous data.

Contribution

The paper proposes a neural network-based CHFL approach that splits the model into two parts for common and unique features, enabling better utilization of heterogeneous data in federated learning.

Findings

CHFL outperforms vanilla HFL on real-world datasets.

Utilizes both shared and client-specific features effectively.

Improves model accuracy in heterogeneous feature scenarios.

Abstract

Federated learning is a promising machine learning technique that enables multiple clients to collaboratively build a model without revealing the raw data to each other. Among various types of federated learning methods, horizontal federated learning (HFL) is the best-studied category and handles homogeneous feature spaces. However, in the case of heterogeneous feature spaces, HFL uses only common features and leaves client-specific features unutilized. In this paper, we propose a HFL method using neural networks named continual horizontal federated learning (CHFL), a continual learning approach to improve the performance of HFL by taking advantage of unique features of each client. CHFL splits the network into two columns corresponding to common features and unique features, respectively. It jointly trains the first column by using common features through vanilla HFL and locally trains the second column by using unique features and leveraging the knowledge of the first one via lateral connections without interfering with the federated training of it. We conduct experiments on various real world datasets and show that CHFL greatly outperforms vanilla HFL that only uses common features and local learning that uses all features that each client has.