Robust Distributional Regression with Automatic Variable Selection

TL;DR

This paper introduces a robust distributional regression method using the generalized normal distribution to handle heavy tails and heteroscedasticity, coupled with a novel penalized estimation for automatic variable selection without extensive tuning.

Contribution

It proposes a new distributional regression approach with a shape parameter for robustness and a smooth information criterion for efficient variable selection, avoiding grid search.

Findings

Effective handling of heavy tails and heteroscedasticity.

Automatic variable selection with a single tuning parameter.

Improved robustness over classical regression methods.

Abstract

Datasets with extreme observations and/or heavy-tailed error distributions are commonly encountered and should be analyzed with careful consideration of these features from a statistical perspective. Small deviations from an assumed model, such as the presence of outliers, can cause classical regression procedures to break down, potentially leading to unreliable inferences. Other distributional deviations, such as heteroscedasticity, can be handled by going beyond the mean and modelling the scale parameter in terms of covariates. We propose a method that accounts for heavy tails and heteroscedasticity through the use of a generalized normal distribution (GND). The GND contains a kurtosis-characterizing shape parameter that moves the model smoothly between the normal distribution and the heavier-tailed Laplace distribution - thus covering both classical and robust regression. A key component of statistical inference is determining the set of covariates that influence the response variable. While correctly accounting for kurtosis and heteroscedasticity is crucial to this endeavour, a procedure for variable selection is still required. For this purpose, we use a novel penalized estimation procedure that avoids the typical computationally demanding grid search for tuning parameters. This is particularly valuable in the distributional regression setting where the location and scale parameters depend on covariates, since the standard approach would have multiple tuning parameters (one for each distributional parameter). We achieve this by using a "smooth information criterion" that can be optimized directly, where the tuning parameters are fixed at log(n) in the BIC case.