Generalized Sparse Additive Model with Unknown Link Function

TL;DR

This paper introduces GSAMUL, a novel sparse additive model that estimates component functions and an unknown link function using B-splines and neural networks, enabling variable selection and interaction detection in high-dimensional data.

Contribution

It proposes a new method combining B-splines and MLPs for simultaneous estimation of link and component functions, with theoretical convergence guarantees.

Findings

Effective in variable selection and interaction detection

Validated on synthetic and real datasets

Converges under theoretical guarantees

Abstract

Generalized additive models (GAM) have been successfully applied to high dimensional data analysis. However, most existing methods cannot simultaneously estimate the link function, the component functions and the variable interaction. To alleviate this problem, we propose a new sparse additive model, named generalized sparse additive model with unknown link function (GSAMUL), in which the component functions are estimated by B-spline basis and the unknown link function is estimated by a multi-layer perceptron (MLP) network. Furthermore, $\ell_{2,1}$-norm regularizer is used for variable selection. The proposed GSAMUL can realize both variable selection and hidden interaction. We integrate this estimation into a bilevel optimization problem, where the data is split into training set and validation set. In theory, we provide the guarantees about the convergence of the approximate procedure. In applications, experimental evaluations on both synthetic and real world data sets consistently validate the effectiveness of the proposed approach.