Ultra high dimensional generalized additive model: Unified Theory and Methods

TL;DR

This paper develops a two-step method for ultra high dimensional generalized additive models, combining group lasso and adaptive group lasso, with theoretical guarantees and numerical validation.

Contribution

It introduces a novel two-step selection and estimation approach for ultra high dimensional GAMs, with proven consistency and improved convergence rates.

Findings

Group lasso effectively selects relevant functions with high probability.

Adaptive group lasso achieves selection consistency and better convergence.

GIC procedure reliably selects the true model.

Abstract

Generalized additive model is a powerful statistical learning and predictive modeling tool that has been applied in a wide range of applications. The need of high-dimensional additive modeling is eminent in the context of dealing with high through-put data such as genetic data analysis. In this article, we studied a two step selection and estimation method for ultra high dimensional generalized additive models. The first step applies group lasso on the expanded bases of the functions. With high probability this selects all nonzero functions without having too much over selection. The second step uses adaptive group lasso with any initial estimators, including the group lasso estimator, that satisfies some regular conditions. The adaptive group lasso estimator is shown to be selection consistent with improved convergence rates. Tuning parameter selection is also discussed and shown to select the true model consistently under GIC procedure. The theoretical properties are supported by extensive numerical study.