Nonlinear variable selection with continuous outcome: a nonparametric incremental forward stagewise approach

TL;DR

This paper introduces a nonparametric incremental forward stagewise method for variable selection in sparse generalized additive models, capable of handling large candidate sets without assuming specific functional forms.

Contribution

It proposes a novel residual adjustment approach called roughening and demonstrates competitive performance against popular machine learning methods.

Findings

Method performs well in simulations

Effective on real datasets

Available in the nlnet R package

Abstract

We present a method of variable selection for the sparse generalized additive model. The method doesn't assume any specific functional form, and can select from a large number of candidates. It takes the form of incremental forward stagewise regression. Given no functional form is assumed, we devised an approach termed roughening to adjust the residuals in the iterations. In simulations, we show the new method is competitive against popular machine learning approaches. We also demonstrate its performance using some real datasets. The method is available as a part of the nlnet package on CRAN https://cran.r-project.org/package=nlnet.