Asymptotic optimality of sparse linear discriminant analysis with arbitrary number of classes

TL;DR

This paper establishes the asymptotic optimality of a broad class of linear classification rules, including sparse LDA methods, for high-dimensional data with any number of classes under multivariate normality.

Contribution

It provides general criteria and convergence rates for asymptotic optimality of linear classifiers with multiple classes in high dimensions, extending previous two-class results.

Findings

Classic LDA and sparse extensions are asymptotically optimal under the new criteria.

Simulation studies confirm the theoretical results across various high-dimensional settings.

The paper offers practical guidelines for assessing classifier optimality in complex, multi-class scenarios.

Abstract

Many sparse linear discriminant analysis (LDA) methods have been proposed to overcome the major problems of the classic LDA in high-dimensional settings. However, the asymptotic optimality results are limited to the case that there are only two classes, which is due to the fact that the classification boundary of LDA is a hyperplane and explicit formulas exist for the classification error in this case. In the situation where there are more than two classes, the classification boundary is usually complicated and no explicit formulas for the classification errors exist. In this paper, we consider the asymptotic optimality in the high-dimensional settings for a large family of linear classification rules with arbitrary number of classes under the situation of multivariate normal distribution. Our main theorem provides easy-to-check criteria for the asymptotic optimality of a general classification rule in this family as dimensionality and sample size both go to infinity and the number of classes is arbitrary. We establish the corresponding convergence rates. The general theory is applied to the classic LDA and the extensions of two recently proposed sparse LDA methods to obtain the asymptotic optimality. We conduct simulation studies on the extended methods in various settings.