HyperFed: Hyperbolic Prototypes Exploration with Consistent Aggregation for Non-IID Data in Federated Learning

TL;DR

HyperFed introduces a hyperbolic prototype-based framework with consistent aggregation to improve federated learning performance on non-IID data by addressing class shift, hierarchical info utilization, and aggregation inconsistency.

Contribution

It proposes a novel hyperbolic prototype exploration and consistent aggregation method to tackle non-IID challenges in federated learning.

Findings

Significant performance improvements on four datasets.

Effective mitigation of non-IID data issues.

Enhanced class and hierarchical information utilization.

Abstract

Federated learning (FL) collaboratively models user data in a decentralized way. However, in the real world, non-identical and independent data distributions (non-IID) among clients hinder the performance of FL due to three issues, i.e., (1) the class statistics shifting, (2) the insufficient hierarchical information utilization, and (3) the inconsistency in aggregating clients. To address the above issues, we propose HyperFed which contains three main modules, i.e., hyperbolic prototype Tammes initialization (HPTI), hyperbolic prototype learning (HPL), and consistent aggregation (CA). Firstly, HPTI in the server constructs uniformly distributed and fixed class prototypes, and shares them with clients to match class statistics, further guiding consistent feature representation for local clients. Secondly, HPL in each client captures the hierarchical information in local data with the supervision of shared class prototypes in the hyperbolic model space. Additionally, CA in the server mitigates the impact of the inconsistent deviations from clients to server. Extensive studies of four datasets prove that HyperFed is effective in enhancing the performance of FL under the non-IID set.