UMEDA: Unified Multi-modal Efficient Data Fusion for Privacy-Preserving Graph Federated Learning via Spectral-Gated Attention and Diffusion-Based Operator Alignment

TL;DR

UMEDA introduces a novel graph federated learning framework that aligns multi-modal sensor data using spectral signal processing, enhancing privacy, accuracy, and efficiency in device-free localization across heterogeneous edge devices.

Contribution

It proposes a spectral-gated attention and diffusion-based operator alignment approach for privacy-preserving multi-modal federated learning, handling heterogeneity and missing modalities effectively.

Findings

UMEDA outperforms state-of-the-art federated baselines in accuracy and convergence.

It maintains formal differential privacy while preserving dominant signal eigendirections.

The method improves communication efficiency especially under high modality heterogeneity.

Abstract

Device-free localization trains models from heterogeneous wireless and visual sensors (e.g., Wi-Fi, LiDAR) distributed across edge devices. Federated learning offers a privacy-respecting framework, but is brittle when clients differ in sensor modality and resolution, when their data distributions drift, and when privacy noise destroys the structural signal needed for localization. We propose UMEDA, a graph federated learning framework in which clients form nodes of a global graph that share a continuous integral operator, and aggregation is reformulated as spectral signal processing on this operator. Each client encodes its local sensors with a linear-attention layer whose kernel spectrum is low-rank filtered, suppressing modality-specific residuals so clients with different sensors align in a common low-rank subspace. The server then aggregates client updates via a diffusion model over the kernel's spectral coefficients, treating updates as discretizations of a shared operator rather than topology-bound weights -- this absorbs varying graph sizes and missing modalities without node-wise correspondence. To balance privacy and utility, we add an anisotropic differential-privacy mechanism that projects noise preferentially into the null space of the signal subspace, preserving dominant eigendirections while ensuring formal $(\epsilon, \delta)$-DP under gradient clipping. On MM-Fi and the RELI11D out-of-distribution benchmark, UMEDA outperforms state-of-the-art federated baselines in accuracy, convergence, and communication efficiency, particularly under high modality heterogeneity and tight privacy budgets.