Unequal Covariance Awareness for Fisher Discriminant Analysis and Its Variants in Classification

TL;DR

This paper introduces a new classification rule for Fisher Discriminant Analysis that accounts for unequal covariance matrices, improving its robustness and performance in practical scenarios.

Contribution

It proposes a novel classification rule that adapts FDA to handle unequal covariance matrices, applicable to various FDA variants, with theoretical analysis and experimental validation.

Findings

Improved classification accuracy over traditional FDA

Effective handling of unequal covariance matrices

Enhanced performance of FDA variants in experiments

Abstract

Fisher Discriminant Analysis (FDA) is one of the essential tools for feature extraction and classification. In addition, it motivates the development of many improved techniques based on the FDA to adapt to different problems or data types. However, none of these approaches make use of the fact that the assumption of equal covariance matrices in FDA is usually not satisfied in practical situations. Therefore, we propose a novel classification rule for the FDA that accounts for this fact, mitigating the effect of unequal covariance matrices in the FDA. Furthermore, since we only modify the classification rule, the same can be applied to many FDA variants, improving these algorithms further. Theoretical analysis reveals that the new classification rule allows the implicit use of the class covariance matrices while increasing the number of parameters to be estimated by a small amount compared to going from FDA to Quadratic Discriminant Analysis. We illustrate our idea via experiments, which show the superior performance of the modified algorithms based on our new classification rule compared to the original ones.