Cross-Silo Heterogeneous Model Federated Multitask Learning

TL;DR

This paper introduces CoFED, a federated learning method that enables cross-silo organizations with different models and tasks to collaboratively train high-quality models while preserving data privacy and intellectual property.

Contribution

The paper proposes CoFED, a novel federated learning approach that handles heterogeneous models, tasks, and training processes using unlabeled data pseudolabeling and cotraining.

Findings

Outperforms existing methods in heterogeneous and non-IID settings

Achieves up to 35% performance improvement

Effective for cross-silo federated learning scenarios

Abstract

Federated learning (FL) is a machine learning technique that enables participants to collaboratively train high-quality models without exchanging their private data. Participants utilizing cross-silo federated learning (CS-FL) settings are independent organizations with different task needs, and they are concerned not only with data privacy but also with independently training their unique models due to intellectual property considerations. Most existing FL methods are incapable of satisfying the above scenarios. In this study, we present a novel federated learning method CoFED based on unlabeled data pseudolabeling via a process known as cotraining. CoFED is a federated learning method that is compatible with heterogeneous models, tasks, and training processes. The experimental results suggest that the proposed method outperforms competing ones. This is especially true for non-independent and identically distributed settings and heterogeneous models, where the proposed method achieves a 35% performance improvement.