High-dimensional generalized linear models and the lasso

TL;DR

This paper establishes nonasymptotic oracle inequalities for high-dimensional generalized linear models using Lasso, covering logistic regression, density estimation, and hinge loss classification, with theoretical guarantees.

Contribution

It introduces a novel nonasymptotic oracle inequality for the empirical risk minimizer with Lasso penalty in high-dimensional GLMs, including various loss functions.

Findings

Oracle inequality proven for high-dimensional GLMs with Lasso

Applicable to logistic regression, density estimation, hinge loss classification

Includes discussion on least squares regression

Abstract

We consider high-dimensional generalized linear models with Lipschitz loss functions, and prove a nonasymptotic oracle inequality for the empirical risk minimizer with Lasso penalty. The penalty is based on the coefficients in the linear predictor, after normalization with the empirical norm. The examples include logistic regression, density estimation and classification with hinge loss. Least squares regression is also discussed.