JDRec: Practical Actor-Critic Framework for Online Combinatorial Recommender System

TL;DR

JDRec introduces a practical actor-critic framework for online combinatorial recommendation, optimizing list generation and evaluation to improve user engagement and platform value.

Contribution

The paper presents a novel actor-critic reinforcement learning framework tailored for online combinatorial recommender systems, addressing efficiency, personalization, and dynamic adaptation.

Findings

Improved click-through rate by 2.6% in online JD recommendation

Enhanced platform value by 5.03% through JDRec

Demonstrated effectiveness in offline and online experiments

Abstract

A combinatorial recommender (CR) system feeds a list of items to a user at a time in the result page, in which the user behavior is affected by both contextual information and items. The CR is formulated as a combinatorial optimization problem with the objective of maximizing the recommendation reward of the whole list. Despite its importance, it is still a challenge to build a practical CR system, due to the efficiency, dynamics, personalization requirement in online environment. In particular, we tear the problem into two sub-problems, list generation and list evaluation. Novel and practical model architectures are designed for these sub-problems aiming at jointly optimizing effectiveness and efficiency. In order to adapt to online case, a bootstrap algorithm forming an actor-critic reinforcement framework is given to explore better recommendation mode in long-term user interaction. Offline and online experiment results demonstrate the efficacy of proposed JDRec framework. JDRec has been applied in online JD recommendation, improving click through rate by 2.6% and synthetical value for the platform by 5.03%. We will publish the large-scale dataset used in this study to contribute to the research community.