Hierarchical clustered multiclass discriminant analysis via cross-validation

TL;DR

This paper introduces a hierarchical clustered multiclass discriminant analysis method that leverages cross-validation to optimize cluster formation, significantly improving prediction accuracy while maintaining computational efficiency.

Contribution

It proposes a novel cluster-based LDA approach using hierarchical clustering guided by cross-validation, with an efficient algorithm for approximate CV computation.

Findings

Achieves high prediction accuracy on artificial and real datasets

Provides fast computation compared to traditional CV-based methods

Demonstrates theoretical and numerical efficiency

Abstract

Linear discriminant analysis (LDA) is a well-known method for multiclass classification and dimensionality reduction. However, in general, ordinary LDA does not achieve high prediction accuracy when observations in some classes are difficult to be classified. This study proposes a novel cluster-based LDA method that significantly improves the prediction accuracy. We adopt hierarchical clustering, and the dissimilarity measure of two clusters is defined by the cross-validation (CV) value. Therefore, clusters are constructed such that the misclassification error rate is minimized. Our approach involves a heavy computational load because the CV value must be computed at each step of the hierarchical clustering algorithm. To address this issue, we develop a regression formulation for LDA and construct an efficient algorithm that computes an approximate value of the CV. The performance of the proposed method is investigated by applying it to both artificial and real datasets. Our proposed method provides high prediction accuracy with fast computation from both numerical and theoretical viewpoints.