Robust Parametric Classification and Variable Selection by a Minimum Distance Criterion

TL;DR

This paper introduces a robust penalized logistic regression method using a minimum distance criterion combined with an Elastic Net penalty, effectively handling outliers and performing variable selection in high-dimensional settings.

Contribution

It proposes a novel combination of minimum distance criterion and Elastic Net penalty for robust logistic regression, addressing outlier influence and variable selection simultaneously.

Findings

Method effectively handles outliers in logistic regression.

Prevents parameter estimates from imploding due to outliers.

Demonstrates good performance in high-dimensional, small sample size scenarios.

Abstract

We investigate a robust penalized logistic regression algorithm based on a minimum distance criterion. Influential outliers are often associated with the explosion of parameter vector estimates, but in the context of standard logistic regression, the bias due to outliers always causes the parameter vector to implode, that is shrink towards the zero vector. Thus, using LASSO-like penalties to perform variable selection in the presence of outliers can result in missed detections of relevant covariates. We show that by choosing a minimum distance criterion together with an Elastic Net penalty, we can simultaneously find a parsimonious model and avoid estimation implosion even in the presence of many outliers in the important small $n$ large $p$ situation. Implementation using an MM algorithm is described and performance evaluated.