On Modeling Long-Term User Engagement from Stochastic Feedback

TL;DR

This paper introduces an efficient reinforcement learning approach for recommender systems that models user engagement directly from data, accounting for randomness in user feedback and termination, demonstrated through real-world experiments.

Contribution

It proposes a novel RL-based method that eliminates the need for storing candidate items and models stochastic user feedback and termination behavior.

Findings

The approach improves user engagement modeling efficiency.

Modeling randomness enhances recommendation performance.

Validated through real-world A/B experiments.

Abstract

An ultimate goal of recommender systems (RS) is to improve user engagement. Reinforcement learning (RL) is a promising paradigm for this goal, as it directly optimizes overall performance of sequential recommendation. However, many existing RL-based approaches induce huge computational overhead, because they require not only the recommended items but also all other candidate items to be stored. This paper proposes an efficient alternative that does not require the candidate items. The idea is to model the correlation between user engagement and items directly from data. Moreover, the proposed approach consider randomness in user feedback and termination behavior, which are ubiquitous for RS but rarely discussed in RL-based prior work. With online A/B experiments on real-world RS, we confirm the efficacy of the proposed approach and the importance of modeling the two types of randomness.