CoRe-Fed: Bridging Collaborative and Representation Fairness via Federated Embedding Distillation

TL;DR

CoRe-Fed introduces a unified federated learning framework that enhances fairness by aligning client representations and adjusting contribution weights, leading to improved model performance and fairness across heterogeneous data sources.

Contribution

It proposes a novel embedding-level regularization and fairness-aware aggregation method to address both representation and collaborative fairness in federated learning.

Findings

Improves fairness metrics across clients.

Enhances model accuracy on diverse datasets.

Reduces representation bias and contribution disparity.

Abstract

With the proliferation of distributed data sources, Federated Learning (FL) has emerged as a key approach to enable collaborative intelligence through decentralized model training while preserving data privacy. However, conventional FL algorithms often suffer from performance disparities across clients caused by heterogeneous data distributions and unequal participation, which leads to unfair outcomes. Specifically, we focus on two core fairness challenges, i.e., representation bias, arising from misaligned client representations, and collaborative bias, stemming from inequitable contribution during aggregation, both of which degrade model performance and generalizability. To mitigate these disparities, we propose CoRe-Fed, a unified optimization framework that bridges collaborative and representation fairness via embedding-level regularization and fairness-aware aggregation. Initially, an alignment-driven mechanism promotes semantic consistency between local and global embeddings to reduce representational divergence. Subsequently, a dynamic reward-penalty-based aggregation strategy adjusts each client's weight based on participation history and embedding alignment to ensure contribution-aware aggregation. Extensive experiments across diverse models and datasets demonstrate that CoRe-Fed improves both fairness and model performance over the state-of-the-art baseline algorithms.