Precision Matrix Regularization in Sufficient Dimension Reduction for Improved Quadratic Discriminant Classification

TL;DR

This paper introduces stabilized SDR (SSDR), a novel method that uses precision matrix shrinkage estimators to improve quadratic discriminant classification, especially in small-sample, high-dimensional settings.

Contribution

The paper proposes a distribution-free SDR method with theoretical guarantees, integrating precision matrix shrinkage to enhance classification stability and accuracy.

Findings

SSDR improves classification accuracy in simulations.

SSDR outperforms existing SDR methods on real datasets.

Theoretical guarantee of preserving classification information.

Abstract

Sufficient dimension reduction (SDR) methods, which often rely on class precision matrices, are widely used in supervised statistical classification problems. However, when class-specific sample sizes are small relative to the original feature-space dimension, precision matrix estimation becomes unstable and, as a result, increases the variability of the linear dimension reduction (LDR) matrix. Ultimately, this fact causes suboptimal supervised classification. To address this problem, we develop a multiclass and distribution-free SDR method, stabilized SDR (SSDR), that employs user-specified precision matrix shrinkage estimators to stabilize the LDR projection matrix and supervised classifier. We establish this technique with the theoretical guarantee of preserving all classification information under the quadratic discriminant analysis (QDA) decision rule. We evaluate multiple precision matrix shrinkage estimators within our proposed SSDR framework through Monte Carlo simulations and applications to real datasets. Our empirical results demonstrate the efficacy of the SSDR method, which generally improves classification accuracy and frequently outperforms several well-established competing SDR methods.