Inference for Large Scale Regression Models with Dependent Errors

TL;DR

This paper introduces GMWMX, a scalable and statistically valid method for inference in large-scale regression models with dependent errors, addressing computational challenges and data complexities like missing data and long-range dependencies.

Contribution

It develops and proves the properties of GMWMX, a novel method for efficient inference in complex, large-scale dependent data settings, improving reliability over existing approximations.

Findings

GMWMX is scalable and stable for large datasets.

It provides reliable inference despite data dependencies and missing data.

Applied examples demonstrate its practical advantages.

Abstract

The exponential growth in data sizes and storage costs has brought considerable challenges to the data science community, requiring solutions to run learning methods on such data. While machine learning has scaled to achieve predictive accuracy in big data settings, statistical inference and uncertainty quantification tools are still lagging. Priority scientific fields collect vast data to understand phenomena typically studied with statistical methods like regression. In this setting, regression parameter estimation can benefit from efficient computational procedures, but the main challenge lies in computing error process parameters with complex covariance structures. Identifying and estimating these structures is essential for inference and often used for uncertainty quantification in machine learning with Gaussian Processes. However, estimating these structures becomes burdensome as data scales, requiring approximations that compromise the reliability of outputs. These approximations are even more unreliable when complexities like long-range dependencies or missing data are present. This work defines and proves the statistical properties of the Generalized Method of Wavelet Moments with Exogenous variables (GMWMX), a highly scalable, stable, and statistically valid method for estimating and delivering inference for linear models using stochastic processes in the presence of data complexities like latent dependence structures and missing data. Applied examples from Earth Sciences and extensive simulations highlight the advantages of the GMWMX.