Federated Learning for Chronic Obstructive Pulmonary Disease Classification with Partial Personalized Attention Mechanism

TL;DR

This paper introduces a novel personalized federated learning framework based on vision transformers for COPD classification from CT images, effectively handling data heterogeneity across multiple medical centers without compromising privacy.

Contribution

It proposes the first PFL framework for ViT tailored to COPD CT classification, with partial personalization of attention heads to improve model performance on heterogeneous data.

Findings

Outperforms existing PFL methods for CNNs on multi-center COPD data

Effectively handles Non-IID data distributions in federated learning

Enhances COPD detection accuracy across diverse medical centers

Abstract

Chronic Obstructive Pulmonary Disease (COPD) is the fourth leading cause of death worldwide. Yet, COPD diagnosis heavily relies on spirometric examination as well as functional airway limitation, which may cause a considerable portion of COPD patients underdiagnosed especially at the early stage. Recent advance in deep learning (DL) has shown their promising potential in COPD identification from CT images. However, with heterogeneous syndromes and distinct phenotypes, DL models trained with CTs from one data center fail to generalize on images from another center. Due to privacy regularizations, a collaboration of distributed CT images into one centralized center is not feasible. Federated learning (FL) approaches enable us to train with distributed private data. Yet, routine FL solutions suffer from performance degradation in the case where COPD CTs are not independent and identically distributed (Non-IID). To address this issue, we propose a novel personalized federated learning (PFL) method based on vision transformer (ViT) for distributed and heterogeneous COPD CTs. To be more specific, we partially personalize some heads in multiheaded self-attention layers to learn the personalized attention for local data and retain the other heads shared to extract the common attention. To the best of our knowledge, this is the first proposal of a PFL framework specifically for ViT to identify COPD. Our evaluation of a dataset set curated from six medical centers shows our method outperforms the PFL approaches for convolutional neural networks.