High-dimensional Measurement Error Models for Lipschitz Loss

TL;DR

This paper introduces a novel high-dimensional measurement error model for Lipschitz loss functions, providing scalable estimators with theoretical guarantees and demonstrating superior performance in biomedical classification tasks.

Contribution

It develops noise-distribution-free estimators for high-dimensional Lipschitz loss models and extends them to scalable Lasso versions with proven statistical guarantees.

Findings

Outperforms existing methods in classification and quantile regression.

Provides finite sample error bounds and sign consistency.

Successfully applied to brain connectivity data for gender classification.

Abstract

Recently emerging large-scale biomedical data pose exciting opportunities for scientific discoveries. However, the ultrahigh dimensionality and non-negligible measurement errors in the data may create difficulties in estimation. There are limited methods for high-dimensional covariates with measurement error, that usually require knowledge of the noise distribution and focus on linear or generalized linear models. In this work, we develop high-dimensional measurement error models for a class of Lipschitz loss functions that encompasses logistic regression, hinge loss and quantile regression, among others. Our estimator is designed to minimize the $L_1$ norm among all estimators belonging to suitable feasible sets, without requiring any knowledge of the noise distribution. Subsequently, we generalize these estimators to a Lasso analog version that is computationally scalable to higher dimensions. We derive theoretical guarantees in terms of finite sample statistical error bounds and sign consistency, even when the dimensionality increases exponentially with the sample size. Extensive simulation studies demonstrate superior performance compared to existing methods in classification and quantile regression problems. An application to a gender classification task based on brain functional connectivity in the Human Connectome Project data illustrates improved accuracy under our approach, and the ability to reliably identify significant brain connections that drive gender differences.