Variable Selection Using a Smooth Information Criterion for Distributional Regression Models

TL;DR

This paper introduces a computationally efficient variable selection method using a smooth information criterion (SIC) within distributional regression models, enabling automatic tuning parameter selection and handling heteroscedastic data.

Contribution

The authors extend the SIC-based variable selection approach to distributional regression, simplifying model selection by avoiding multiple tuning parameters and improving computational efficiency.

Findings

SIC-based method automatically selects tuning parameters in distributional regression.

The approach effectively models heteroscedastic data with multiple distributional parameters.

Computational advantages over traditional cross-validation and BIC methods.

Abstract

Modern variable selection procedures make use of penalization methods to execute simultaneous model selection and estimation. A popular method is the LASSO (least absolute shrinkage and selection operator), the use of which requires selecting the value of a tuning parameter. This parameter is typically tuned by minimizing the cross-validation error or Bayesian information criterion (BIC) but this can be computationally intensive as it involves fitting an array of different models and selecting the best one. In contrast with this standard approach, we have developed a procedure based on the so-called "smooth IC" (SIC) in which the tuning parameter is automatically selected in one step. We also extend this model selection procedure to the distributional regression framework, which is more flexible than classical regression modelling. Distributional regression, also known as multiparameter regression (MPR), introduces flexibility by taking account of the effect of covariates through multiple distributional parameters simultaneously, e.g., mean and variance. These models are useful in the context of normal linear regression when the process under study exhibits heteroscedastic behaviour. Reformulating the distributional regression estimation problem in terms of penalized likelihood enables us to take advantage of the close relationship between model selection criteria and penalization. Utilizing the SIC is computationally advantageous, as it obviates the issue of having to choose multiple tuning parameters.