Regret Minimization and Statistical Inference in Online Decision Making with High-dimensional Covariates

TL;DR

This paper develops methods for regret minimization and statistical inference in high-dimensional online decision-making, balancing exploration and exploitation, with theoretical guarantees and practical validation.

Contribution

It introduces a combined framework using bandit algorithms and debiased estimators for simultaneous regret minimization and inference in high-dimensional settings.

Findings

Achieves $O(T^{1/2})$ regret or inference under a margin condition.

Pure-greedy bandit with debiased estimator attains $O( ext{log} T)$ regret and $O(T^{1/2})$ inference.

Sample mean estimator provides valid inference for the optimal policy.

Abstract

This paper investigates regret minimization, statistical inference, and their interplay in high-dimensional online decision-making based on the sparse linear context bandit model. We integrate the $\varepsilon$-greedy bandit algorithm for decision-making with a hard thresholding algorithm for estimating sparse bandit parameters and introduce an inference framework based on a debiasing method using inverse propensity weighting. Under a margin condition, our method achieves either $O(T^{1/2})$ regret or classical $O(T^{1/2})$-consistent inference, indicating an unavoidable trade-off between exploration and exploitation. If a diverse covariate condition holds, we demonstrate that a pure-greedy bandit algorithm, i.e., exploration-free, combined with a debiased estimator based on average weighting can simultaneously achieve optimal $O(\log T)$ regret and $O(T^{1/2})$-consistent inference. We also show that a simple sample mean estimator can provide valid inference for the optimal policy's value. Numerical simulations and experiments on Warfarin dosing data validate the effectiveness of our methods.