Sparse additive models in high dimensions with wavelets

TL;DR

This paper introduces a computationally efficient method for high-dimensional additive models using wavelets, enabling effective variable selection and prediction without cross-validation, with superior false discovery and true positive rates.

Contribution

It presents a novel wavelet-based convex optimization approach for variable selection in high-dimensional additive models, avoiding cross-validation and improving FDR-TPR trade-offs.

Findings

Outperforms existing methods in FDR and TPR.

Handles high-dimensional data effectively.

Provides accurate predictions in real data applications.

Abstract

In multivariate regression, when covariates are numerous, it is often reasonable to assume that only a small number of them has predictive information. In some medical applications for instance, it is believed that only a few genes out of thousands are responsible for cancers. In that case, the aim is not only to propose a good fit, but also to select the relevant covariates (genes). We propose to perform model selection with additive models in high dimensions (sample size and number of covariates). Our approach is computationally efficient thanks to fast wavelet transforms, it does not rely on cross validation, and it solves a convex optimization problem for a prescribed penalty parameter, called the quantile universal threshold. We also propose a second rule based on Stein unbiased risk estimation geared towards prediction. We use Monte Carlo simulations and real data to compare various methods based on false discovery rate (FDR), true positive rate (TPR) and mean squared error. Our approach is the only one to handle high dimensions, and has the best FDR--TPR trade-off.