Sparse Additive Model using Symmetric Nonnegative Definite Smoothers

TL;DR

This paper presents an adaptive sparse backfitting algorithm for high-dimensional Sparse Additive Models that guarantees convergence and improves performance over previous methods, with proven variable selection consistency.

Contribution

The paper introduces a new block coordinate descent algorithm for SpAM using symmetric non-negative definite smoothers, ensuring convergence and variable selection consistency.

Findings

Outperforms previous sparse backfitting algorithms in fitting accuracy

Demonstrates superior prediction performance on synthetic and real data

Proves variable selection consistency under certain conditions

Abstract

We introduce a new algorithm, called adaptive sparse backfitting algorithm, for solving high dimensional Sparse Additive Model (SpAM) utilizing symmetric, non-negative definite smoothers. Unlike the previous sparse backfitting algorithm, our method is essentially a block coordinate descent algorithm that guarantees to converge to the optimal solution. It bridges the gap between the population backfitting algorithm and that of the data version. We also prove variable selection consistency under suitable conditions. Numerical studies on both synthesis and real data are conducted to show that adaptive sparse backfitting algorithm outperforms previous sparse backfitting algorithm in fitting and predicting high dimensional nonparametric models.