High dimensional stochastic linear contextual bandit with missing covariates

TL;DR

This paper analyzes the impact of missing covariates on regret in high-dimensional stochastic linear bandit problems, providing bounds and practical algorithms for experimental design with incomplete data.

Contribution

It introduces a new regret bound for linear bandits with missing covariates and proposes an algorithm for sequential experimental design under such conditions.

Findings

Regret increases at most by the inverse of the minimum observation probability.

The proposed algorithm effectively handles missing data in high-dimensional settings.

Application demonstrated in gene expression data collection for DNA probes.

Abstract

Recent works in bandit problems adopted lasso convergence theory in the sequential decision-making setting. Even with fully observed contexts, there are technical challenges that hinder the application of existing lasso convergence theory: 1) proving the restricted eigenvalue condition under conditionally sub-Gaussian noise and 2) accounting for the dependence between the context variables and the chosen actions. This paper studies the effect of missing covariates on regret for stochastic linear bandit algorithms. Our work provides a high-probability upper bound on the regret incurred by the proposed algorithm in terms of covariate sampling probabilities, showing that the regret degrades due to missingness by at most $\zeta_{min}^2$, where $\zeta_{min}$ is the minimum probability of observing covariates in the context vector. We illustrate our algorithm for the practical application of experimental design for collecting gene expression data by a sequential selection of class discriminating DNA probes.