# A Federated Recommendation System with a Dual-Layer Multi-Head Attention Network and Regularization Strategy

## TL;DR

This paper introduces a new federated recommendation system that improves privacy and performance by using a dual-layer attention network and regularization.

## Contribution

FedDMR introduces a dual-layer multi-head attention network and regularization strategy to enhance federated recommendation systems.

## Key findings

- FedDMR improves AUC and precision by 2.63% compared to existing federated recommendation baselines.
- The dual-layer attention network enriches user feature representations using local interaction data.
- Regularization reduces overfitting and improves model generalizability in decentralized settings.

## Abstract

Federated recommendation (FedRec) aims to provide effective recommendation services while preserving user privacy. However, in a federated setting, a single user cannot access other users’ interaction data. With limited local interactions, existing FedRec models struggle to fully exploit interaction information to learn users’ preferences. Moreover, training recommendation models in decentralized FedRec scenarios suffer from a risk of overfitting. To address the above issues, we propose a federated recommendation system with a dual-layer multi-head attention network and regularization strategy (FedDMR). First, FedDMR initializes clients’ local recommendation models. Subsequently, clients perform local training based on their private data. Our dual-layer multi-head attention network is designed to perform attention-weighted interactions on user and item embeddings, progressively capturing local interaction information and generating interaction-aware embeddings, thereby enriching users’ feature representations for modeling personalized preferences. Then, a regularization strategy is employed to guide updates to clients’ models by constraining their deviation from the global parameters, which effectively mitigates overfitting caused by limited local data and enhances the generalizability of the models. Finally, the server aggregates the clients’ uploaded parameters for this round. The entire training process is implemented through the federated learning framework. Experimental results on three datasets demonstrate that FedDMR achieves an average improvement of 2.63% in AUC and precision compared to the recent federated recommendation baselines.