Bayesian Non-stationary Linear Bandits for Large-Scale Recommender Systems

TL;DR

This paper introduces a scalable, non-stationary linear bandit algorithm using random projections and exponential weighting, improving recommendation accuracy in high-dimensional, dynamic environments.

Contribution

It proposes a novel Thompson sampling-based policy for high-dimensional, non-stationary bandits that reduces computational complexity while maintaining low regret.

Findings

The algorithm achieves lower regret compared to state-of-the-art methods.

It effectively handles high-dimensional, non-stationary data in real-time.

Numerical experiments on real datasets validate its efficiency and robustness.

Abstract

Taking advantage of contextual information can potentially boost the performance of recommender systems. In the era of big data, such side information often has several dimensions. Thus, developing decision-making algorithms to cope with such a high-dimensional context in real time is essential. That is specifically challenging when the decision-maker has a variety of items to recommend. In addition, changes in items' popularity or users' preferences can hinder the performance of the deployed recommender system due to a lack of robustness to distribution shifts in the environment. In this paper, we build upon the linear contextual multi-armed bandit framework to address this problem. We develop a decision-making policy for a linear bandit problem with high-dimensional feature vectors, a large set of arms, and non-stationary reward-generating processes. Our Thompson sampling-based policy reduces the dimension of feature vectors using random projection and uses exponentially increasing weights to decrease the influence of past observations with time. Our proposed recommender system employs this policy to learn the users' item preferences online while minimizing runtime. We prove a regret bound that scales as a factor of the reduced dimension instead of the original one. To evaluate our proposed recommender system numerically, we apply it to three real-world datasets. The theoretical and numerical results demonstrate the effectiveness of our proposed algorithm in making a trade-off between computational complexity and regret performance compared to the state-of-the-art.