High dimensional discriminant rules with shrinkage estimators of the covariance matrix and mean vector

TL;DR

This paper explores advanced shrinkage estimators for covariance matrices and mean vectors in high-dimensional linear discriminant analysis, providing theoretical insights and empirical evaluations on simulated and real datasets.

Contribution

It introduces non-parametric shrinkage methods for covariance and mean estimation in high-dimensional LDA, with new theoretical performance results and comprehensive simulation and real data analysis.

Findings

NPEB method's performance is theoretically characterized.

Shrinkage estimators improve classification accuracy in high dimensions.

Empirical results demonstrate effectiveness on gene expression and EEG data.

Abstract

Linear discriminant analysis (LDA) is a typical method for classification problems with large dimensions and small samples. There are various types of LDA methods that are based on the different types of estimators for the covariance matrices and mean vectors. In this paper, we consider shrinkage methods based on a non-parametric approach. For the precision matrix, methods based on the sparsity structure or data splitting are examined. Regarding the estimation of mean vectors, Non-parametric Empirical Bayes (NPEB) methods and Non-parametric Maximum Likelihood Estimation (NPMLE) methods, also known as f-modeling and g-modeling, respectively, are adopted. The performance of linear discriminant rules based on combined estimation strategies of the covariance matrix and mean vectors are analyzed in this study. Particularly, the study presents a theoretical result on the performance of the NPEB method and compares it with previous studies. Simulation studies with various covariance matrices and mean vector structures are conducted to evaluate the methods discussed in this paper. Furthermore, real data examples such as gene expressions and EEG data are also presented