Context Uncertainty in Contextual Bandits with Applications to Recommender Systems

TL;DR

This paper introduces Recurrent Exploration Networks (REN), a novel neural network model that balances relevance and exploration in recommender systems by accounting for uncertainty in user and item representations, leading to improved long-term performance.

Contribution

The paper proposes REN, a new recurrent neural network that jointly learns representations and explores effectively under uncertainty, with proven regret bounds and superior empirical results.

Findings

REN achieves sublinear regret under representation uncertainty.

REN outperforms state-of-the-art models on synthetic and real datasets.

REN effectively balances exploration and relevance in recommendations.

Abstract

Recurrent neural networks have proven effective in modeling sequential user feedbacks for recommender systems. However, they usually focus solely on item relevance and fail to effectively explore diverse items for users, therefore harming the system performance in the long run. To address this problem, we propose a new type of recurrent neural networks, dubbed recurrent exploration networks (REN), to jointly perform representation learning and effective exploration in the latent space. REN tries to balance relevance and exploration while taking into account the uncertainty in the representations. Our theoretical analysis shows that REN can preserve the rate-optimal sublinear regret even when there exists uncertainty in the learned representations. Our empirical study demonstrates that REN can achieve satisfactory long-term rewards on both synthetic and real-world recommendation datasets, outperforming state-of-the-art models.