PNCS:Power-Norm Cosine Similarity for Diverse Client Selection in Federated Learning

TL;DR

This paper introduces PNCS, a novel client selection method in federated learning that captures gradient correlations and higher-order moments, improving convergence and accuracy in heterogeneous data scenarios.

Contribution

The paper proposes Power-Norm Cosine Similarity (PNCS) for better client selection in federated learning, addressing data heterogeneity and gradient correlation issues.

Findings

PNCS improves convergence speed over existing methods.

Experiments show higher accuracy with PNCS across data partitions.

Diverse client selection enhances model robustness.

Abstract

Federated Learning (FL) has emerged as a powerful paradigm for leveraging diverse datasets from multiple sources while preserving data privacy by avoiding centralized storage. However, many existing approaches fail to account for the intricate gradient correlations between remote clients, a limitation that becomes especially problematic in data heterogeneity scenarios. In this work, we propose a novel FL framework utilizing Power-Norm Cosine Similarity (PNCS) to improve client selection for model aggregation. By capturing higher-order gradient moments, PNCS addresses non-IID data challenges, enhancing convergence speed and accuracy. Additionally, we introduce a simple algorithm ensuring diverse client selection through a selection history queue. Experiments with a VGG16 model across varied data partitions demonstrate consistent improvements over state-of-the-art methods.