Enhancing Federated Quadruplet Learning: Stochastic Client Selection and Embedding Stability Analysis

TL;DR

This paper introduces FedQuad, a federated learning method that improves model generalisation by enforcing intra-class compactness and inter-class separation, with extensive evaluation on image datasets.

Contribution

The paper proposes FedQuad, a novel federated learning approach that explicitly minimizes intra-class and maximizes inter-class distances to enhance robustness against data heterogeneity.

Findings

FedQuad outperforms existing baselines on CIFAR-10, CIFAR-100, and Tiny-ImageNet.

The approach effectively mitigates representation misalignment caused by data heterogeneity.

Analysis shows metric learning improves representation stability in federated settings.

Abstract

Federated Learning (FL) enables decentralised model training across distributed clients without requiring data centralisation. However, the generalisation performance of the global model is usually degraded by data heterogeneity across clients, particularly under limited data availability and class imbalance. To address this challenge, we propose FedQuad, a novel method that explicitly enforces minimising intra-class representations while enabling inter-class splits across clients. By jointly minimising distances between positive pairs and maximising distances between negative pairs, the proposed approach mitigates representation misalignment introduced during model aggregation. We evaluate our method on CIFAR-10, CIFAR-100, and Tiny-ImageNet under diverse non-IID settings and varying numbers of clients, demonstrating consistent improvements over existing baselines. Additionally, we provide a comprehensive analysis of metric learning-based approaches in both centralised and federated environments, highlighting their effectiveness in alleviating representation collapse under heterogeneous data distributions.