Quantum Federated Learning for Multimodal Data: A Modality-Agnostic Approach

TL;DR

This paper introduces a novel quantum federated learning framework capable of handling multiple data modalities using quantum entanglement, with mechanisms to maintain performance despite missing modalities, demonstrating significant accuracy improvements.

Contribution

It presents the first multimodal quantum federated learning approach with an innovative MMA mechanism for robustness against missing modalities.

Findings

Achieves 6.84% accuracy improvement in IID data

Achieves 7.25% accuracy improvement in non-IID data

Demonstrates robustness to missing modalities in quantum federated learning

Abstract

Quantum federated learning (QFL) has been recently introduced to enable a distributed privacy-preserving quantum machine learning (QML) model training across quantum processors (clients). Despite recent research efforts, existing QFL frameworks predominantly focus on unimodal systems, limiting their applicability to real-world tasks that often naturally involve multiple modalities. To fill this significant gap, we present for the first time a novel multimodal approach specifically tailored for the QFL setting with the intermediate fusion using quantum entanglement. Furthermore, to address a major bottleneck in multimodal QFL, where the absence of certain modalities during training can degrade model performance, we introduce a Missing Modality Agnostic (MMA) mechanism that isolates untrained quantum circuits, ensuring stable training without corrupted states. Simulation results demonstrate that the proposed multimodal QFL method with MMA yields an improvement in accuracy of 6.84% in independent and identically distributed (IID) and 7.25% in non-IID data distributions compared to the state-of-the-art methods.