Stochastic Lipschitz continuity for high dimensional Lasso with multiple linear covariate structures or hidden linear covariates

TL;DR

This paper develops stochastic Lipschitz continuity results for complex high-dimensional regression models, including those with multiple covariate structures or hidden covariates, and applies these to derive bounds on Lasso estimation errors.

Contribution

It introduces stochastic Lipschitz continuity results for advanced high-dimensional models and provides new bounds on Lasso estimation errors in these settings.

Findings

Derived stochastic Lipschitz continuity for complex models

Established bounds on Lasso estimation error

Used Rademacher complexity for multivariate comparison

Abstract

Two extensions of generalized linear models are considered. In the first one, response variables depend on multiple linear combinations of covariates. In the second one, only response variables are observed while the linear covariates are missing. We derive stochastic Lipschitz continuity results for the loss functions involved in the regression problems and apply them to get bounds on estimation error for Lasso. Multivariate comparison results on Rademacher complexity are obtained as tools to establish the stochastic Lipschitz continuity results.