Stochastic Online Instrumental Variable Regression: Regrets for Endogeneity and Bandit Feedback

TL;DR

This paper introduces an online instrumental variable regression method, O2SLS, to handle endogeneity in stochastic online learning, and develops OFUL-IV, a bandit algorithm that achieves near-optimal regret even with endogeneity.

Contribution

It proposes O2SLS for online IV regression and OFUL-IV for linear bandits with endogeneity, addressing a gap in existing methods that assume exogeneity.

Findings

O2SLS achieves $ ilde{O}(d_x d_z ext{log}^2 T)$ identification and $ ilde{O}(eta ext{sqrt}(d_z T))$ regret.

O2SLS matches stochastic ridge regret under exogeneity with $ ilde{O}(d_x^2 ext{log}^2 T)$ regret.

OFUL-IV attains $ ilde{O}( ext{sqrt}(d_x d_z T))$ regret, matching lower bounds under exogeneity.

Abstract

Endogeneity, i.e. the dependence of noise and covariates, is a common phenomenon in real data due to omitted variables, strategic behaviours, measurement errors etc. In contrast, the existing analyses of stochastic online linear regression with unbounded noise and linear bandits depend heavily on exogeneity, i.e. the independence of noise and covariates. Motivated by this gap, we study the over- and just-identified Instrumental Variable (IV) regression, specifically Two-Stage Least Squares, for stochastic online learning, and propose to use an online variant of Two-Stage Least Squares, namely O2SLS. We show that O2SLS achieves $\mathcal O(d_{x}d_{z}\log^2 T)$ identification and $\widetilde{\mathcal O}(\gamma \sqrt{d_{z} T})$ oracle regret after $T$ interactions, where $d_{x}$ and $d_{z}$ are the dimensions of covariates and IVs, and $\gamma$ is the bias due to endogeneity. For $\gamma=0$, i.e. under exogeneity, O2SLS exhibits $\mathcal O(d_{x}^2 \log^2 T)$ oracle regret, which is of the same order as that of the stochastic online ridge. Then, we leverage O2SLS as an oracle to design OFUL-IV, a stochastic linear bandit algorithm to tackle endogeneity. OFUL-IV yields $\widetilde{\mathcal O}(\sqrt{d_{x}d_{z}T})$ regret that matches the regret lower bound under exogeneity. For different datasets with endogeneity, we experimentally show efficiencies of O2SLS and OFUL-IV.