Quadratic Discriminant Analysis by Projection

TL;DR

This paper introduces a new hybrid dimension reduction method for quadratic discriminant analysis that effectively handles heteroscedastic data in moderate dimensions, combining the robustness of LDA with the flexibility of QDA.

Contribution

It proposes a novel approach to estimate the optimal 1D subspace for QDA, improving stability and performance in moderate-dimensional heteroscedastic data.

Findings

The method is consistent in estimation.

It outperforms LDA, QDA, and RDA in simulations.

It performs well on real data examples.

Abstract

Discriminant analysis, including linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA), is a popular approach to classification problems. It is well known that LDA is suboptimal to analyze heteroscedastic data, for which QDA would be an ideal tool. However, QDA is less helpful when the number of features in a data set is moderate or high, and LDA and its variants often perform better due to their robustness against dimensionality. In this work, we introduce a new dimension reduction and classification method based on QDA. In particular, we define and estimate the optimal one-dimensional (1D) subspace for QDA, which is a novel hybrid approach to discriminant analysis. The new method can handle data heteroscedasticity with number of parameters equal to that of LDA. Therefore, it is more stable than the standard QDA and works well for data in moderate dimensions. We show an estimation consistency property of our method, and compare it with LDA, QDA, regularized discriminant analysis (RDA) and a few other competitors by simulated and real data examples.