Robust adaptive Lasso in high-dimensional logistic regression

TL;DR

This paper introduces a robust adaptive Lasso method for high-dimensional logistic regression that improves stability and accuracy in noisy data, especially in genomic disease classification.

Contribution

It proposes a novel robust estimator using density power divergence and adaptive LASSO penalties, with theoretical analysis and empirical validation.

Findings

Enhanced robustness over existing methods

Significant improvement in classification accuracy

Effective gene selection in real datasets

Abstract

Penalized logistic regression is extremely useful for binary classification with large number of covariates (higher than the sample size), having several real life applications, including genomic disease classification. However, the existing methods based on the likelihood loss function are sensitive to data contamination and other noise and, hence, robust methods are needed for stable and more accurate inference. In this paper, we propose a family of robust estimators for sparse logistic models utilizing the popular density power divergence based loss function and the general adaptively weighted LASSO penalties. We study the local robustness of the proposed estimators through its influence function and also derive its oracle properties and asymptotic distribution. With extensive empirical illustrations, we demonstrate the significantly improved performance of our proposed estimators over the existing ones with particular gain in robustness. Our proposal is finally applied to analyse four different real datasets for cancer classification, obtaining robust and accurate models, that simultaneously performs gene selection and patient classification.