Sharpness-Aware Minimization for Generalized Embedding Learning in Federated Recommendation

TL;DR

This paper introduces FedRecGEL, a federated recommendation framework that employs sharpness-aware minimization to stabilize the learning of generalized item embeddings, improving recommendation accuracy in heterogeneous data environments.

Contribution

It proposes a novel item-centered, multi-task learning approach with sharpness-aware minimization to enhance generalized embedding learning in federated recommendation systems.

Findings

Significant performance improvements on four datasets.

Effective stabilization of training process.

Enhanced generalization of item embeddings.

Abstract

Federated recommender systems enable collaborative model training while keeping user interaction data local and sharing only essential model parameters, thereby mitigating privacy risks. However, existing methods overlook a critical issue, i.e., the stable learning of a generalized item embedding throughout the federated recommender system training process. Item embedding plays a central role in facilitating knowledge sharing across clients. Yet, under the cross-device setting, local data distributions exhibit significant heterogeneity and sparsity, exacerbating the difficulty of learning generalized embeddings. These factors make the stable learning of generalized item embeddings both indispensable for effective federated recommendation and inherently difficult to achieve. To fill this gap, we propose a new federated recommendation framework, named Federated Recommendation with Generalized Embedding Learning (FedRecGEL). We reformulate the federated recommendation problem from an item-centered perspective and cast it as a multi-task learning problem, aiming to learn generalized embeddings throughout the training procedure. Based on theoretical analysis, we employ sharpness-aware minimization to address the generalization problem, thereby stabilizing the training process and enhancing recommendation performance. Extensive experiments on four datasets demonstrate the effectiveness of FedRecGEL in significantly improving federated recommendation performance. Our code is available at https://github.com/anonymifish/FedRecGEL.