Sheaf-Based Decentralized Multimodal Learning for Next-Generation Wireless Communication Systems

TL;DR

This paper introduces Sheaf-DMFL, a decentralized multimodal learning framework using sheaf theory to improve collaboration among edge devices with diverse sensory data in wireless communication systems.

Contribution

It proposes a novel sheaf-theoretic approach for decentralized multimodal learning, enabling effective collaboration among heterogeneous devices with different modalities.

Findings

Sheaf-DMFL outperforms traditional federated learning in multimodal scenarios.

Sheaf-DMFL-Att improves learning by capturing cross-modality correlations.

Theoretical convergence guarantees are established for Sheaf-DMFL-Att.

Abstract

In large-scale communication systems, increasingly complex scenarios require more intelligent collaboration among edge devices collecting various multimodal sensory data to achieve a more comprehensive understanding of the environment and improve decision-making accuracy. However, conventional federated learning (FL) algorithms typically consider unimodal datasets, require identical model architectures, and fail to leverage the rich information embedded in multimodal data, limiting their applicability to real-world scenarios with diverse modalities and varying client capabilities. To address this issue, we propose Sheaf-DMFL, a novel decentralized multimodal learning framework leveraging sheaf theory to enhance collaboration among devices with diverse modalities. Specifically, each client has a set of local feature encoders for its different modalities, whose outputs are concatenated before passing through a task-specific layer. While encoders for the same modality are trained collaboratively across clients, we capture the intrinsic correlations among clients' task-specific layers using a sheaf-based structure. To further enhance learning capability, we propose an enhanced algorithm named Sheaf-DMFL-Att, which tailors the attention mechanism within each client to capture correlations among different modalities. A rigorous convergence analysis of Sheaf-DMFL-Att is provided, establishing its theoretical guarantees. Extensive simulations are conducted on real-world link blockage prediction and mmWave beamforming scenarios, demonstrate the superiority of the proposed algorithms in such heterogeneous wireless communication systems.