Sparse Fisher's discriminant analysis with thresholded linear constraints

TL;DR

This paper introduces a new sparse Fisher's LDA method with thresholded linear constraints that achieves asymptotic optimality in high-dimensional multiclass classification without estimating the covariance matrix.

Contribution

It establishes asymptotic consistency and optimality for the proposed sparse Fisher's LDA in high dimensions for any number of classes, using a novel thresholding approach.

Findings

Achieves asymptotic optimality in high-dimensional multiclass classification.

Avoids covariance matrix estimation via thresholding.

Demonstrates effectiveness on multivariate functional data.

Abstract

Various regularized linear discriminant analysis (LDA) methods have been proposed to address the problems of the classic methods in high-dimensional settings. Asymptotic optimality has been established for some of these methods in high dimension when there are only two classes. A major difficulty in proving asymptotic optimality for multiclass classification is that the classification boundary is typically complicated and no explicit formula for classification error generally exists when the number of classes is greater than two. For the Fisher's LDA, one additional difficulty is that the covariance matrix is also involved in the linear constraints. The main purpose of this paper is to establish asymptotic consistency and asymptotic optimality for our sparse Fisher's LDA with thresholded linear constraints in the high-dimensional settings for arbitrary number of classes. To address the first difficulty above, we provide asymptotic optimality and the corresponding convergence rates in high-dimensional settings for a large family of linear classification rules with arbitrary number of classes, and apply them to our method. To overcome the second difficulty, we propose a thresholding approach to avoid the estimate of the covariance matrix. We apply the method to the classification problems for multivariate functional data through the wavelet transformations.