Multi-level Feedback Web Links Selection Problem: Learning and Optimization

TL;DR

This paper models the web links selection problem as a constrained multi-armed bandit with multi-level feedback, proposing an algorithm that learns link structures to maximize revenue while satisfying feedback constraints.

Contribution

It introduces the first model for links selection with multi-level feedback as a constrained bandit problem and provides an effective algorithm with theoretical guarantees.

Findings

The multi-level feedback structure of web links can be learned from real datasets.

The proposed LExp algorithm outperforms state-of-the-art bandit algorithms in links selection tasks.

The algorithm achieves sub-linear regret and constraint violation bounds.

Abstract

Selecting the right web links for a website is important because appropriate links not only can provide high attractiveness but can also increase the website's revenue. In this work, we first show that web links have an intrinsic \emph{multi-level feedback structure}. For example, consider a $2$-level feedback web link: the $1$st level feedback provides the Click-Through Rate (CTR) and the $2$nd level feedback provides the potential revenue, which collectively produce the compound $2$-level revenue. We consider the context-free links selection problem of selecting links for a homepage so as to maximize the total compound $2$-level revenue while keeping the total $1$st level feedback above a preset threshold. We further generalize the problem to links with $n~(n\ge2)$-level feedback structure. The key challenge is that the links' multi-level feedback structures are unobservable unless the links are selected on the homepage. To our best knowledge, we are the first to model the links selection problem as a constrained multi-armed bandit problem and design an effective links selection algorithm by learning the links' multi-level structure with provable \emph{sub-linear} regret and violation bounds. We uncover the multi-level feedback structures of web links in two real-world datasets. We also conduct extensive experiments on the datasets to compare our proposed \textbf{LExp} algorithm with two state-of-the-art context-free bandit algorithms and show that \textbf{LExp} algorithm is the most effective in links selection while satisfying the constraint.