A Dirichlet Process Functional Approach to Heteroscedastic-Consistent Covariance Estimation

TL;DR

This paper introduces a novel Dirichlet process-based approach to covariance estimation in regression, linking Bayesian nonparametrics with heteroscedasticity-consistent methods and ridge regression, enhancing robustness and interpretability.

Contribution

It develops a new functional framework connecting Dirichlet process priors with weighted least squares and heteroscedasticity-consistent covariance estimation, including a generalized ridge regression for high-dimensional data.

Findings

Provides a Bayesian interpretation of heteroscedastic-consistent covariance estimators.

Introduces a new ridge regression estimator handling multicollinearity and high-dimensionality.

Demonstrates effectiveness through simulations and real data analysis.

Abstract

The mixture of Dirichlet process (MDP) defines a flexible prior distribution on the space of probability measures. This study shows that ordinary least-squares (OLS) estimator, as a functional of the MDP posterior distribution, has posterior mean given by weighted least-squares (WLS), and has posterior covariance matrix given by the (weighted) heteroscedastic-consistent sandwich estimator. This is according to a pairs bootstrap distribution approximation of the posterior, using a P\'olya urn scheme. Also, when the MDP prior baseline distribution is specified as a product of independent probability measures, this WLS solution provides a new type of generalized ridge regression estimator which can handle multicollinear or singular design matrices even when the number of covariates exceeds the sample size, and which shrinks the coefficient estimates of irrelevant covariates towards zero, thus useful for nonlinear regressions. Also, this MDP/OLS functional methodology can be extended to methods for analyzing the sensitivity of the heteroscedasticity-consistent causal effect size over a range of hidden biases due to missing covariates omitted from the regression, and more generally extended to a Vibration of Effects analysis. The methodology is illustrated through the analysis of simulated and real data sets. Overall, this study establishes new connections between Dirichlet process functional inference, the bootstrap, consistent sandwich covariance estimation, ridge shrinkage regression, WLS, and sensitivity analysis, to provide regression methodology useful for inferences of the mean dependent response.