MH-pFLID: Model Heterogeneous personalized Federated Learning via Injection and Distillation for Medical Data Analysis

TL;DR

This paper introduces MH-pFLID, a federated learning framework designed for medical data that handles system heterogeneity and privacy concerns by using a lightweight messenger model and efficient information injection and distillation.

Contribution

The paper proposes a novel federated learning paradigm that manages heterogeneous client systems without requiring public datasets, enhancing privacy and efficiency in medical data analysis.

Findings

Effective handling of system heterogeneity in federated learning.

Reduces reliance on public datasets for knowledge distillation.

Improves privacy and resource efficiency in medical data analysis.

Abstract

Federated learning is widely used in medical applications for training global models without needing local data access. However, varying computational capabilities and network architectures (system heterogeneity), across clients pose significant challenges in effectively aggregating information from non-independently and identically distributed (non-IID) data. Current federated learning methods using knowledge distillation require public datasets, raising privacy and data collection issues. Additionally, these datasets require additional local computing and storage resources, which is a burden for medical institutions with limited hardware conditions. In this paper, we introduce a novel federated learning paradigm, named Model Heterogeneous personalized Federated Learning via Injection and Distillation (MH-pFLID). Our framework leverages a lightweight messenger model that carries concentrated information to collect the information from each client. We also develop a set of receiver and transmitter modules to receive and send information from the messenger model, so that the information could be injected and distilled with efficiency.