FedKDX: Federated Learning with Negative Knowledge Distillation for Enhanced Healthcare AI Systems

TL;DR

FedKDX introduces a federated learning framework utilizing Negative Knowledge Distillation to improve healthcare AI models by capturing both positive and non-target information, enhancing accuracy, convergence, and privacy in decentralized medical data settings.

Contribution

The paper presents FedKDX, a novel federated learning approach that incorporates Negative Knowledge Distillation alongside traditional methods to better handle data heterogeneity in healthcare applications.

Findings

Achieves up to 2.53% accuracy improvement over state-of-the-art methods.

Demonstrates faster convergence and improved performance on non-IID healthcare data.

Supports privacy-preserving medical AI with theoretical and empirical validation.

Abstract

This paper introduces FedKDX, a federated learning framework that addresses limitations in healthcare AI through Negative Knowledge Distillation (NKD). Unlike existing approaches that focus solely on positive knowledge transfer, FedKDX captures both target and non-target information to improve model generalization in healthcare applications. The framework integrates multiple knowledge transfer techniques--including traditional knowledge distillation, contrastive learning, and NKD--within a unified architecture that maintains privacy while reducing communication costs. Through experiments on healthcare datasets (SLEEP, UCI-HAR, and PAMAP2), FedKDX demonstrates improved accuracy (up to 2.53% over state-of-the-art methods), faster convergence, and better performance on non-IID data distributions. Theoretical analysis supports NKD's contribution to addressing statistical heterogeneity in distributed healthcare data. The approach shows promise for privacy-sensitive medical applications under regulatory frameworks like HIPAA and GDPR, offering a balanced solution between performance and practical implementation requirements in decentralized healthcare settings. The code and model are available at https://github.com/phamdinhdat-ai/Fed_2024.