Multicategory vertex discriminant analysis for high-dimensional data

TL;DR

This paper introduces an enhanced vertex discriminant analysis method that performs simultaneous classification and variable selection in high-dimensional data using lasso and Euclidean penalties, improving model efficiency and prediction accuracy.

Contribution

It develops a penalized version of vertex discriminant analysis incorporating lasso and Euclidean penalties for better variable selection and classification in high-dimensional contexts.

Findings

Penalized VDA effectively eliminates irrelevant predictors.

The method accelerates estimation with cyclic coordinate descent.

Tests show improved prediction accuracy on real and simulated data.

Abstract

In response to the challenges of data mining, discriminant analysis continues to evolve as a vital branch of statistics. Our recently introduced method of vertex discriminant analysis (VDA) is ideally suited to handle multiple categories and an excess of predictors over training cases. The current paper explores an elaboration of VDA that conducts classification and variable selection simultaneously. Adding lasso ($\ell_1$-norm) and Euclidean penalties to the VDA loss function eliminates unnecessary predictors. Lasso penalties apply to each predictor coefficient separately; Euclidean penalties group the collective coefficients of a single predictor. With these penalties in place, cyclic coordinate descent accelerates estimation of all coefficients. Our tests on simulated and benchmark real data demonstrate the virtues of penalized VDA in model building and prediction in high-dimensional settings.