Calibrated zero-norm regularized LS estimator for high-dimensional error-in-variables regression

TL;DR

This paper introduces a calibrated zero-norm regularized least squares estimator for high-dimensional error-in-variables regression, improving variable selection and estimation accuracy under measurement errors.

Contribution

It proposes a novel CaZnRLS estimator with a calibrated loss and multi-stage convex relaxation, providing theoretical guarantees and superior empirical performance.

Findings

Achieves sign consistency after finite steps.

Provides error bounds under restricted eigenvalue condition.

Outperforms CoCoLasso and NCL in numerical tests.

Abstract

This paper is concerned with high-dimensional error-in-variables regression that aims at identifying a small number of important interpretable factors for corrupted data from many applications where measurement errors or missing data can not be ignored. Motivated by CoCoLasso due to Datta and Zou \cite{Datta16} and the advantage of the zero-norm regularized LS estimator over Lasso for clean data, we propose a calibrated zero-norm regularized LS (CaZnRLS) estimator by constructing a calibrated least squares loss with a positive definite projection of an unbiased surrogate for the covariance matrix of covariates, and use the multi-stage convex relaxation approach to compute the CaZnRLS estimator. Under a restricted eigenvalue condition on the true matrix of covariates, we derive the $\ell_2$-error bound of every iterate and establish the decreasing of the error bound sequence, and the sign consistency of the iterates after finite steps. The statistical guarantees are also provided for the CaZnRLS estimator under two types of measurement errors. Numerical comparisons with CoCoLasso and NCL (the nonconvex Lasso proposed by Poh and Wainwright \cite{Loh11}) demonstrate that CaZnRLS not only has the comparable or even better relative RSME but also has the least number of incorrect predictors identified.