Enhancing Bandit Algorithms with LLMs for Time-varying User Preferences in Streaming Recommendations

TL;DR

This paper introduces HyperBandit+, a novel bandit algorithm that incorporates time-aware hypernetworks and LLM-assisted warm-starts to adapt to evolving user preferences in streaming recommendations, improving exploration and overall performance.

Contribution

The paper proposes HyperBandit+, integrating a time-aware hypernetwork and LLM-based warm-start to better handle time-varying preferences and enhance early exploration in streaming recommender systems.

Findings

HyperBandit+ outperforms state-of-the-art baselines in real-world datasets.

Theoretical regret bounds are established for the proposed method.

Low-rank factorization reduces training complexity without sacrificing performance.

Abstract

In real-world streaming recommender systems, user preferences evolve dynamically over time. Existing bandit-based methods treat time merely as a timestamp, neglecting its explicit relationship with user preferences and leading to suboptimal performance. Moreover, online learning methods often suffer from inefficient exploration-exploitation during the early online phase. To address these issues, we propose HyperBandit+, a novel contextual bandit policy that integrates a time-aware hypernetwork to adapt to time-varying user preferences and employs a large language model-assisted warm-start mechanism (LLM Start) to enhance exploration-exploitation efficiency in the early online phase. Specifically, HyperBandit+ leverages a neural network that takes time features as input and generates parameters for estimating time-varying rewards by capturing the correlation between time and user preferences. Additionally, the LLM Start mechanism employs multi-step data augmentation to simulate realistic interaction data for effective offline learning, providing warm-start parameters for the bandit policy in the early online phase. To meet real-time streaming recommendation demands, we adopt low-rank factorization to reduce hypernetwork training complexity. Theoretically, we rigorously establish a sublinear regret upper bound that accounts for both the hypernetwork and the LLM warm-start mechanism. Extensive experiments on real-world datasets demonstrate that HyperBandit+ consistently outperforms state-of-the-art baselines in terms of accumulated rewards.