# Cascading Non-Stationary Bandits: Online Learning to Rank in the Non-Stationary Cascade Model

## TL;DR

This paper introduces cascading non-stationary bandits for online ranking in environments with changing user preferences, proposing algorithms with near-optimal regret bounds and validating on real web search data.

## Contribution

It develops the first algorithms for non-stationary cascading bandits, providing theoretical regret bounds and empirical validation.

## Key findings

- Algorithms CascadeDUCB and CascadeSWUCB achieve near-optimal regret bounds.
- Both algorithms perform well on real-world web search data.
- Theoretical analysis matches lower bounds up to logarithmic factors.

## Abstract

Non-stationarity appears in many online applications such as web search and advertising. In this paper, we study the online learning to rank problem in a non-stationary environment where user preferences change abruptly at an unknown moment in time. We consider the problem of identifying the K most attractive items and propose cascading non-stationary bandits, an online learning variant of the cascading model, where a user browses a ranked list from top to bottom and clicks on the first attractive item. We propose two algorithms for solving this non-stationary problem: CascadeDUCB and CascadeSWUCB. We analyze their performance and derive gap-dependent upper bounds on the n-step regret of these algorithms. We also establish a lower bound on the regret for cascading non-stationary bandits and show that both algorithms match the lower bound up to a logarithmic factor. Finally, we evaluate their performance on a real-world web search click dataset.