Online Policy Learning and Inference by Matrix Completion

TL;DR

This paper introduces a matrix completion bandit approach for online decision-making without covariates, combining policy learning with online estimation and debiasing, demonstrated on parking data.

Contribution

It proposes a novel two-phase policy learning method using matrix completion and online gradient descent for covariate-free decisions.

Findings

Outperforms benchmark policies in parking data application

Develops an online debiasing inference method with asymptotic normality

Balances policy accuracy and regret through a two-phase design

Abstract

Is it possible to make online decisions when personalized covariates are unavailable? We take a collaborative-filtering approach for decision-making based on collective preferences. By assuming low-dimensional latent features, we formulate the covariate-free decision-making problem as a matrix completion bandit. We propose a policy learning procedure that combines an $\varepsilon$-greedy policy for decision-making with an online gradient descent algorithm for bandit parameter estimation. Our novel two-phase design balances policy learning accuracy and regret performance. For policy inference, we develop an online debiasing method based on inverse propensity weighting and establish its asymptotic normality. Our methods are applied to data from the San Francisco parking pricing project, revealing intriguing discoveries and outperforming the benchmark policy.