Contextual Bandits with Sparse Data in Web setting

TL;DR

This paper reviews current methods for handling sparse data with contextual bandits in web applications, categorizing techniques and providing an updated overview of the field from 2017 to 2020.

Contribution

It offers a comprehensive categorization of existing methods for sparse data in contextual bandits, facilitating method selection and future evaluation in web settings.

Findings

Identified 19 method articles and 2 review articles from 2017-2020.

Categorized methods into five main groups for addressing sparse data.

Provided an overview of problem areas and techniques for future research.

Abstract

This paper is a scoping study to identify current methods used in handling sparse data with contextual bandits in web settings. The area is highly current and state of the art methods are identified. The years 2017-2020 are investigated, and 19 method articles are identified, and two review articles. Five categories of methods are described, making it easy to choose how to address sparse data using contextual bandits with a method available for modification in the specific setting of concern. In addition, each method has multiple techniques to choose from for future evaluation. The problem areas are also mentioned that each article covers. An overall updated understanding of sparse data problems using contextual bandits in web settings is given. The identified methods are policy evaluation (off-line and on-line) , hybrid-method, model representation (clusters and deep neural networks), dimensionality reduction, and simulation.