Least Squares Estimation Using Sketched Data with Heteroskedastic Errors

TL;DR

This paper demonstrates that using random projection sketches in least squares regression effectively neutralizes heteroskedasticity, simplifying inference without sacrificing accuracy, even with endogenous covariates in instrumental variables settings.

Contribution

It shows that sketched least squares estimates behave as if errors are homoskedastic, enabling simpler inference methods in large-scale regressions with heteroskedastic errors.

Findings

Random projections lead to asymptotically normal estimates with homoskedastic variance.

The results hold for both exogenous and endogenous covariates in IV estimation.

Sketching simplifies inference procedures like F tests for instrument relevance.

Abstract

Researchers may perform regressions using a sketch of data of size $m$ instead of the full sample of size $n$ for a variety of reasons. This paper considers the case when the regression errors do not have constant variance and heteroskedasticity robust standard errors would normally be needed for test statistics to provide accurate inference. We show that estimates using data sketched by random projections will behave `as if' the errors were homoskedastic. Estimation by random sampling would not have this property. The result arises because the sketched estimates in the case of random projections can be expressed as degenerate $U$-statistics, and under certain conditions, these statistics are asymptotically normal with homoskedastic variance. We verify that the conditions hold not only in the case of least squares regression when the covariates are exogenous, but also in instrumental variables estimation when the covariates are endogenous. The result implies that inference, including first-stage F tests for instrument relevance, can be simpler than the full sample case if the sketching scheme is appropriately chosen.