Accurate Inference for Adaptive Linear Models

TL;DR

This paper introduces a $ extbf{W}$-decorrelation method to correct biases in adaptive linear regression estimators, enabling accurate inference despite sequential dependence in data collection.

Contribution

The paper develops a general bias-to-variance transformation technique for adaptive linear models that requires minimal information about the data collection policy.

Findings

Bounds finite-sample bias and variance of the $ extbf{W}$-estimator.

Provides asymptotically correct confidence intervals using a novel martingale CLT.

Demonstrates empirical improvements in bandit and time series settings.

Abstract

Estimators computed from adaptively collected data do not behave like their non-adaptive brethren. Rather, the sequential dependence of the collection policy can lead to severe distributional biases that persist even in the infinite data limit. We develop a general method -- $\mathbf{W}$-decorrelation -- for transforming the bias of adaptive linear regression estimators into variance. The method uses only coarse-grained information about the data collection policy and does not need access to propensity scores or exact knowledge of the policy. We bound the finite-sample bias and variance of the $\mathbf{W}$-estimator and develop asymptotically correct confidence intervals based on a novel martingale central limit theorem. We then demonstrate the empirical benefits of the generic $\mathbf{W}$-decorrelation procedure in two different adaptive data settings: the multi-armed bandit and the autoregressive time series.