D2RLIR : an improved and diversified ranking function in interactive recommendation systems based on deep reinforcement learning

TL;DR

This paper introduces D2RLIR, a deep reinforcement learning-based interactive recommendation system that enhances ranking efficiency and diversity by combining Actor-Critic architecture, similarity search, and diversity algorithms, validated on MovieLens.

Contribution

It presents a novel recommendation framework integrating Actor-Critic RL, similarity search, and diversity optimization, addressing efficiency and diversity issues in interactive systems.

Findings

Improved recommendation diversity and relevance on MovieLens dataset.

Enhanced ranking efficiency using Spotify's ANNoy algorithm.

Effective modeling of user interactions with positional encoding.

Abstract

Recently, interactive recommendation systems based on reinforcement learning have been attended by researchers due to the consider recommendation procedure as a dynamic process and update the recommendation model based on immediate user feedback, which is neglected in traditional methods. The existing works have two significant drawbacks. Firstly, inefficient ranking function to produce the Top-N recommendation list. Secondly, focusing on recommendation accuracy and inattention to other evaluation metrics such as diversity. This paper proposes a deep reinforcement learning based recommendation system by utilizing Actor-Critic architecture to model dynamic users' interaction with the recommender agent and maximize the expected long-term reward. Furthermore, we propose utilizing Spotify's ANNoy algorithm to find the most similar items to generated action by actor-network. After that, the Total Diversity Effect Ranking algorithm is used to generate the recommendations concerning relevancy and diversity. Moreover, we apply positional encoding to compute representations of the user's interaction sequence without using sequence-aligned recurrent neural networks. Extensive experiments on the MovieLens dataset demonstrate that our proposed model is able to generate a diverse while relevance recommendation list based on the user's preferences.