GraphPL: Leveraging GNN for Efficient and Robust Modalities Imputation in Patchwork Learning

TL;DR

GraphPL introduces a graph neural network-based approach for efficient, robust modality imputation in patchwork learning, outperforming existing methods on benchmarks and real-world health data.

Contribution

It proposes a novel GNN-based framework that effectively integrates all observed modalities for patchwork learning, addressing limitations of prior methods.

Findings

GraphPL achieves state-of-the-art performance on benchmark datasets.

GraphPL learns strong features for downstream tasks like disease prediction.

The method remains robust with noisy modality inputs.

Abstract

Current research on distributed multi-modal learning typically assumes that clients can access complete information across all modalities, which may not hold in practice. In this paper, we explore patchwork learning, in which the modalities available to different clients vary, and the objective is to impute the missing modalities for each client in an unsupervised manner. Existing methods are shown not to fully utilize the modality information as they tend to rely on only a subset of the observed modalities. To address this issue, we propose GraphPL, which combines graph neural networks with patchwork learning to flexibly integrate all observed modalities and remains robust with noisy inputs. Experimental results show that GraphPL achieves SOTA performance on benchmark datasets. Our results on real-world distributed electronic health record dataset show GraphPL learns strong downstream features and enables tasks like disease prediction via superior modality imputation.