Bayesian Models for Joint Selection of Features and Auto-Regressive Lags: Theory and Applications in Environmental and Financial Forecasting

TL;DR

This paper introduces a Bayesian approach for selecting relevant features and lagged variables in time series models with autocorrelated errors, improving prediction accuracy and model interpretability in environmental and financial applications.

Contribution

It develops a hierarchical Bayesian model with spike-and-slab priors and an efficient two-stage MCMC algorithm, providing theoretical guarantees for variable selection consistency in high-dimensional settings.

Findings

Enhanced variable selection accuracy in simulations

Lower mean squared prediction error in real data applications

Robustness to autocorrelated noise compared to existing methods

Abstract

We develop a Bayesian framework for variable selection in linear regression with autocorrelated errors, accommodating lagged covariates and autoregressive structures. This setting occurs in time series applications where responses depend on contemporaneous or past explanatory variables and persistent stochastic shocks, including financial modeling, hydrological forecasting, and meteorological applications requiring temporal dependency capture. Our methodology uses hierarchical Bayesian models with spike-and-slab priors to simultaneously select relevant covariates and lagged error terms. We propose an efficient two-stage MCMC algorithm separating sampling of variable inclusion indicators and model parameters to address high-dimensional computational challenges. Theoretical analysis establishes posterior selection consistency under mild conditions, even when candidate predictors grow exponentially with sample size, common in modern time series with many potential lagged variables. Through simulations and real applications (groundwater depth prediction, S&P 500 log returns modeling), we demonstrate substantial gains in variable selection accuracy and predictive performance. Compared to existing methods, our framework achieves lower MSPE, improved true model component identification, and greater robustness with autocorrelated noise, underscoring practical utility for model interpretation and forecasting in autoregressive settings.