One-shot Robust Federated Learning of Independent Component Analysis

TL;DR

This paper introduces a robust federated ICA method using geometric median aggregation and k-means clustering, effectively handling client heterogeneity and permutation ambiguity in distributed settings.

Contribution

It proposes a novel one-shot aggregation framework combining geometric median and k-means clustering to address permutation ambiguity and heterogeneity in federated ICA.

Findings

Method remains effective in highly heterogeneous scenarios.

Theoretical analysis of error bounds and clustering misclassification.

Simulation results validate robustness across diverse settings.

Abstract

This paper investigates a general robust one-shot aggregation framework for distributed and federated Independent Component Analysis (ICA) problem. We propose a geometric median-based aggregation algorithm that leverages $k$-means clustering to resolve the permutation ambiguity in local client estimations. Our method first performs k-means to partition client-provided estimators into clusters and then aggregates estimators within each cluster using the geometric median. This approach provably remains effective even in highly heterogeneous scenarios where at most half of the clients can observe only a minimal number of samples. The key theoretical contribution lies in the combined analysis of the geometric median's error bound-aided by sample quantiles-and the maximum misclustering rates of the aforementioned solution of $k$-means. The effectiveness of the proposed approach is further supported by simulation studies conducted under various heterogeneous settings.