Canonical correlation regression with noisy data

TL;DR

This paper introduces a spectral regularization approach for instrumental variable regression with noisy, high-dimensional data, providing theoretical bounds and practical guidance for optimal estimation.

Contribution

It develops a novel two-stage least squares estimator using spectral regularization to handle noisy covariates and instruments, with proven optimality bounds.

Findings

Derived upper and lower bounds on estimation error.

Proved the method's optimality in noisy data settings.

Provided practical guidance on spectral regularization choices.

Abstract

We study instrumental variable regression in data rich environments. The goal is to estimate a linear model from many noisy covariates and many noisy instruments. Our key assumption is that true covariates and true instruments are repetitive, though possibly different in nature; they each reflect a few underlying factors, however those underlying factors may be misaligned. We analyze a family of estimators based on two stage least squares with spectral regularization: canonical correlations between covariates and instruments are learned in the first stage, which are used as regressors in the second stage. As a theoretical contribution, we derive upper and lower bounds on estimation error, proving optimality of the method with noisy data. As a practical contribution, we provide guidance on which types of spectral regularization to use in different regimes.