Feature Selection for Vertex Discriminant Analysis

TL;DR

This paper introduces a sparse vertex discriminant analysis (VDA) method that directly incorporates feature sparsity constraints, enabling effective multiclass classification with controlled feature selection, especially in high-dimensional data.

Contribution

The paper presents a novel sparse VDA approach using proximal distance algorithms to directly control feature sparsity, improving model interpretability and performance.

Findings

Sparse VDA effectively selects relevant features in high-dimensional data.

The method achieves competitive classification accuracy with fewer features.

Applications to kernel-based VDA demonstrate versatility.

Abstract

We revisit vertex discriminant analysis (VDA) from the perspective of proximal distance algorithms. By specifying sparsity sets as constraints that directly control the number of active features, VDA is able to fit multiclass classifiers with no more than $k$ active features. We combine our sparse VDA approach with repeated cross validation to fit classifiers across the full range of model sizes on a given dataset. Our numerical examples demonstrate that grappling with sparsity directly is an attractive approach to model building in high-dimensional settings. Applications to kernel-based VDA are also considered.