FedSlate:A Federated Deep Reinforcement Learning Recommender System

TL;DR

FedSlate introduces a federated reinforcement learning approach for recommendation systems that preserves user privacy while effectively learning long-term user engagement across multiple platforms.

Contribution

The paper presents FedSlate, a novel federated reinforcement learning algorithm that extends recommendation systems to multi-platform scenarios without sharing sensitive user data.

Findings

FedSlate outperforms baseline models in various environments.

It enables cross-platform learning where traditional methods fail.

The approach maintains user privacy while optimizing long-term engagement.

Abstract

Reinforcement learning methods have been used to optimize long-term user engagement in recommendation systems. However, existing reinforcement learning-based recommendation systems do not fully exploit the relevance of individual user behavior across different platforms. One potential solution is to aggregate data from various platforms in a centralized location and use the aggregated data for training. However, this approach raises economic and legal concerns, including increased communication costs and potential threats to user privacy. To address these challenges, we propose \textbf{FedSlate}, a federated reinforcement learning recommendation algorithm that effectively utilizes information that is prohibited from being shared at a legal level. We employ the SlateQ algorithm to assist FedSlate in learning users' long-term behavior and evaluating the value of recommended content. We extend the existing application scope of recommendation systems from single-user single-platform to single-user multi-platform and address cross-platform learning challenges by introducing federated learning. We use RecSim to construct a simulation environment for evaluating FedSlate and compare its performance with state-of-the-art benchmark recommendation models. Experimental results demonstrate the superior effects of FedSlate over baseline methods in various environmental settings, and FedSlate facilitates the learning of recommendation strategies in scenarios where baseline methods are completely inapplicable. Code is available at \textit{https://github.com/TianYaDY/FedSlate}.