Stochastic Optimization Algorithms for Instrumental Variable Regression with Streaming Data

TL;DR

This paper introduces online stochastic algorithms for instrumental variable regression that operate efficiently on streaming data without matrix inversions, providing convergence guarantees and outperforming minimax-based methods.

Contribution

It proposes a fully online, matrix-inversion-free algorithm for instrumental variable regression with streaming data, with proven convergence rates and advantages over minimax reformulations.

Findings

Algorithms achieve convergence rates of O(log T/T) and O(1/T^{1-ι})

Method avoids modeling confounder-instrument relationships

Numerical experiments validate theoretical results

Abstract

We develop and analyze algorithms for instrumental variable regression by viewing the problem as a conditional stochastic optimization problem. In the context of least-squares instrumental variable regression, our algorithms neither require matrix inversions nor mini-batches and provides a fully online approach for performing instrumental variable regression with streaming data. When the true model is linear, we derive rates of convergence in expectation, that are of order $\mathcal{O}(\log T/T)$ and $\mathcal{O}(1/T^{1-\iota})$ for any $\iota>0$, respectively under the availability of two-sample and one-sample oracles, respectively, where $T$ is the number of iterations. Importantly, under the availability of the two-sample oracle, our procedure avoids explicitly modeling and estimating the relationship between confounder and the instrumental variables, demonstrating the benefit of the proposed approach over recent works based on reformulating the problem as minimax optimization problems. Numerical experiments are provided to corroborate the theoretical results.