Group selection and shrinkage: Structured sparsity for semiparametric additive models

TL;DR

This paper introduces structured sparse estimators for semiparametric additive models that handle overlapping groups, improving modeling flexibility and accuracy in applications like economic and traffic data analysis.

Contribution

The work develops a novel optimization framework for nonconvex regularization that allows arbitrary overlapping groups in structured sparsity estimation.

Findings

Improved estimation accuracy on synthetic data.

Effective modeling of supermarket foot traffic and economic recessions.

Demonstrated scalability and practical utility of the algorithms.

Abstract

Sparse regression and classification estimators that respect group structures have application to an assortment of statistical and machine learning problems, from multitask learning to sparse additive modeling to hierarchical selection. This work introduces structured sparse estimators that combine group subset selection with shrinkage. To accommodate sophisticated structures, our estimators allow for arbitrary overlap between groups. We develop an optimization framework for fitting the nonconvex regularization surface and present finite-sample error bounds for estimation of the regression function. As an application requiring structure, we study sparse semiparametric additive modeling, a procedure that allows the effect of each predictor to be zero, linear, or nonlinear. For this task, the new estimators improve across several metrics on synthetic data compared to alternatives. Finally, we demonstrate their efficacy in modeling supermarket foot traffic and economic recessions using many predictors. These demonstrations suggest sparse semiparametric additive models, fit using the new estimators, are an excellent compromise between fully linear and fully nonparametric alternatives. All of our algorithms are made available in the scalable implementation grpsel.