Enhanced variable selection for boosting sparser and less complex models in distributional copula regression

TL;DR

This paper improves variable selection in distributional copula regression models by combining probing, stability selection, and a new deselection method, leading to sparser, more interpretable models without sacrificing predictive accuracy.

Contribution

It introduces a novel deselection approach and combines it with existing methods to enhance variable selection in high-dimensional distributional copula regression models.

Findings

All proposed methods reduce false positives in variable selection.

Stability selection and deselection maintain predictive performance.

Deselection is more scalable to large datasets.

Abstract

Structured additive distributional copula regression allows to model the joint distribution of multivariate outcomes by relating all distribution parameters to covariates. Estimation via statistical boosting enables accounting for high-dimensional data and incorporating data-driven variable selection, both of which are useful given the complexity of the model class. However, as known from univariate (distributional) regression, the standard boosting algorithm tends to select too many variables with minor importance, particularly in settings with large sample sizes, leading to complex models with difficult interpretation. To counteract this behavior and to avoid selecting base-learners with only a negligible impact, we combined the ideas of probing, stability selection and a new deselection approach with statistical boosting for distributional copula regression. In a simulation study and an application to the joint modelling of weight and length of newborns, we found that all proposed methods enhance variable selection by reducing the number of false positives. However, only stability selection and the deselection approach yielded similar predictive performance to classical boosting. Finally, the deselection approach is better scalable to larger datasets and led to a competitive predictive performance, which we further illustrated in a genomic cohort study from the UK Biobank by modelling the joint genetic predisposition for two phenotypes.