Sparse-group boosting -- Unbiased group and variable selection

TL;DR

This paper introduces a boosting framework that enforces sparsity within and between groups of covariates, mimicking sparse group lasso properties, and demonstrates its effectiveness through simulations and real data.

Contribution

It develops a novel sparse group boosting method combining component-wise and group-wise gradient boosting with adjusted degrees of freedom, improving variable selection and prediction.

Findings

Sparse group boosting achieves less biased variable selection.

It offers higher predictive accuracy compared to component-wise boosting.

The method effectively controls within- and between-group sparsity.

Abstract

In the presence of grouped covariates, we propose a framework for boosting that allows to enforce sparsity within and between groups. By using component-wise and group-wise gradient boosting at the same time with adjusted degrees of freedom, a model with similar properties as the sparse group lasso can be fitted through boosting. We show that within-group and between-group sparsity can be controlled by a mixing parameter and discuss similarities and differences to the mixing parameter in the sparse group lasso. With simulations, gene data as well as agricultural data we show the effectiveness and predictive competitiveness of this estimator. The data and simulations suggest, that in the presence of grouped variables the use of sparse group boosting is associated with less biased variable selection and higher predictability compared to component-wise boosting. Additionally, we propose a way of reducing bias in component-wise boosting through the degrees of freedom.