An Efficient Approach to Sparse Linear Discriminant Analysis

TL;DR

This paper introduces a novel, efficient sparse Linear Discriminant Analysis method based on penalized Optimal Scoring, which produces highly sparse models with strong predictive performance, especially suitable for high-dimensional data like gene expression.

Contribution

It proposes a new formulation of sparse LDA using penalized Optimal Scoring with group-Lasso, offering exact equivalence to penalized LDA and improved efficiency.

Findings

Produces highly sparse models without losing accuracy

Efficient for medium to large variable datasets

Suitable for gene expression data analysis

Abstract

We present a novel approach to the formulation and the resolution of sparse Linear Discriminant Analysis (LDA). Our proposal, is based on penalized Optimal Scoring. It has an exact equivalence with penalized LDA, contrary to the multi-class approaches based on the regression of class indicator that have been proposed so far. Sparsity is obtained thanks to a group-Lasso penalty that selects the same features in all discriminant directions. Our experiments demonstrate that this approach generates extremely parsimonious models without compromising prediction performances. Besides prediction, the resulting sparse discriminant directions are also amenable to low-dimensional representations of data. Our algorithm is highly efficient for medium to large number of variables, and is thus particularly well suited to the analysis of gene expression data.