Breaking the Aggregation Bottleneck in Federated Recommendation: A Personalized Model Merging Approach

TL;DR

This paper identifies the aggregation bottleneck in federated recommendation systems caused by heterogeneity and proposes FedEM, a method that elastically merges global and local models to enhance personalization and performance.

Contribution

The paper provides the first theoretical analysis of the aggregation bottleneck in federated recommendation and introduces FedEM, which improves personalization by merging models without extra mechanisms.

Findings

FedEM outperforms state-of-the-art baselines in real-world datasets.

Theoretical insights explain how aggregation affects personalization.

FedEM effectively balances global and local model merging.

Abstract

Federated recommendation (FR) facilitates collaborative training by aggregating local models from massive devices, enabling client-specific personalization while ensuring privacy. However, we empirically and theoretically demonstrate that server-side aggregation can undermine client-side personalization, leading to suboptimal performance, which we term the aggregation bottleneck. This issue stems from the inherent heterogeneity across numerous clients in FR, which drives the globally aggregated model to deviate from local optima. To this end, we propose FedEM, which elastically merges the global and local models to compensate for impaired personalization. Unlike existing personalized federated recommendation (pFR) methods, FedEM (1) investigates the aggregation bottleneck in FR through theoretical insights, rather than relying on heuristic analysis; (2) leverages off-the-shelf local models rather than designing additional mechanisms to boost personalization. Extensive experiments on real-world datasets demonstrate that our method preserves client personalization during collaborative training, outperforming state-of-the-art baselines.