An MDP-based Recommender System
Guy Shani, Ronen I. Brafman, David Heckerman

TL;DR
This paper proposes modeling recommender systems as Markov decision processes to account for long-term effects of recommendations, using an n-gram model to generate the initial MDP for improved user behavior prediction.
Contribution
It introduces an MDP framework for recommender systems and develops an n-gram based initial model that outperforms existing predictive models.
Findings
The n-gram model achieves higher predictive accuracy than existing models.
An MDP-based recommender system effectively incorporates long-term user effects.
The approach is validated on real user data.
Abstract
Typical Recommender systems adopt a static view of the recommendation process and treat it as a prediction problem. We argue that it is more appropriate to view the problem of generating recommendations as a sequential decision problem and, consequently, that Markov decision processes (MDP) provide a more appropriate model for Recommender systems. MDPs introduce two benefits: they take into account the long-term effects of each recommendation, and they take into account the expected value of each recommendation. To succeed in practice, an MDP-based Recommender system must employ a strong initial model; and the bulk of this paper is concerned with the generation of such a model. In particular, we suggest the use of an n-gram predictive model for generating the initial MDP. Our n-gram model induces a Markov-chain model of user behavior whose predictive accuracy is greater than that of…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsRecommender Systems and Techniques · Topic Modeling · Advanced Graph Neural Networks
