The Mismatch Principle: The Generalized Lasso Under Large Model Uncertainties

TL;DR

This paper introduces the mismatch principle, a theoretical framework that explains the robustness of the generalized Lasso in high-dimensional, uncertain, and semi-parametric estimation tasks across various models and data conditions.

Contribution

It provides a general theoretical error bound for the generalized Lasso under broad conditions, extending its applicability to complex, uncertain, and non-linear models without specific observation assumptions.

Findings

The mismatch principle offers a unified approach to error analysis.

Generalized Lasso is robust against model misspecifications and non-linear distortions.

Theoretical guarantees apply to diverse high-dimensional estimation problems.

Abstract

We study the estimation capacity of the generalized Lasso, i.e., least squares minimization combined with a (convex) structural constraint. While Lasso-type estimators were originally designed for noisy linear regression problems, it has recently turned out that they are in fact robust against various types of model uncertainties and misspecifications, most notably, non-linearly distorted observation models. This work provides more theoretical evidence for this somewhat astonishing phenomenon. At the heart of our analysis stands the mismatch principle, which is a simple recipe to establish theoretical error bounds for the generalized Lasso. The associated estimation guarantees are of independent interest and are formulated in a fairly general setup, permitting arbitrary sub-Gaussian data, possibly with strongly correlated feature designs; in particular, we do not assume a specific observation model which connects the input and output variables. Although the mismatch principle is conceived based on ideas from statistical learning theory, its actual application area are (high-dimensional) estimation tasks for semi-parametric models. In this context, the benefits of the mismatch principle are demonstrated for a variety of popular problem classes, such as single-index models, generalized linear models, and variable selection. Apart from that, our findings are also relevant to recent advances in quantized and distributed compressed sensing.