Improving Learning of New Diseases through Knowledge-Enhanced Initialization for Federated Adapter Tuning

TL;DR

This paper introduces FedKEI, a federated learning framework that uses knowledge transfer and clustering to quickly adapt foundation models to new medical diseases while preserving privacy.

Contribution

FedKEI is a novel method that leverages cross-client and cross-task knowledge transfer with clustering and bi-level optimization for rapid adaptation in federated healthcare models.

Findings

FedKEI outperforms existing methods in adapting to new diseases.

Effective knowledge transfer improves model personalization.

Demonstrated on dermatology, chest X-ray, and retinal OCT datasets.

Abstract

In healthcare, federated learning (FL) is a widely adopted framework that enables privacy-preserving collaboration among medical institutions. With large foundation models (FMs) demonstrating impressive capabilities, using FMs in FL through cost-efficient adapter tuning has become a popular approach. Given the rapidly evolving healthcare environment, it is crucial for individual clients to quickly adapt to new tasks or diseases by tuning adapters while drawing upon past experiences. In this work, we introduce Federated Knowledge-Enhanced Initialization (FedKEI), a novel framework that leverages cross-client and cross-task transfer from past knowledge to generate informed initializations for learning new tasks with adapters. FedKEI begins with a global clustering process at the server to generalize knowledge across tasks, followed by the optimization of aggregation weights across clusters (inter-cluster weights) and within each cluster (intra-cluster weights) to personalize knowledge transfer for each new task. To facilitate more effective learning of the inter- and intra-cluster weights, we adopt a bi-level optimization scheme that collaboratively learns the global intra-cluster weights across clients and optimizes the local inter-cluster weights toward each client's task objective. Extensive experiments on three benchmark datasets of different modalities, including dermatology, chest X-rays, and retinal OCT, demonstrate FedKEI's advantage in adapting to new diseases compared to state-of-the-art methods.