Heterogeneous Collaborative Learning for Personalized Healthcare Analytics via Messenger Distillation

TL;DR

This paper introduces SQMD, a novel framework for personalized healthcare analytics that enables heterogeneous devices to collaboratively distill knowledge asynchronously without requiring identical models, improving personalization and reliability.

Contribution

The paper presents a new messenger distillation approach that leverages a reference dataset and dynamic collaboration graphs for heterogeneous, asynchronous healthcare analytics.

Findings

SQMD outperforms existing methods on real-life datasets.

It effectively handles heterogeneity and asynchrony in healthcare device collaboration.

The framework enhances personalization and model reliability.

Abstract

In this paper, we propose a Similarity-Quality-based Messenger Distillation (SQMD) framework for heterogeneous asynchronous on-device healthcare analytics. By introducing a preloaded reference dataset, SQMD enables all participant devices to distill knowledge from peers via messengers (i.e., the soft labels of the reference dataset generated by clients) without assuming the same model architecture. Furthermore, the messengers also carry important auxiliary information to calculate the similarity between clients and evaluate the quality of each client model, based on which the central server creates and maintains a dynamic collaboration graph (communication graph) to improve the personalization and reliability of SQMD under asynchronous conditions. Extensive experiments on three real-life datasets show that SQMD achieves superior performance.