A Harmonic Mean Linear Discriminant Analysis for Robust Image Classification

TL;DR

This paper introduces a harmonic mean-based Linear Discriminant Analysis method called MCDA, which improves image classification by better handling small between-class distances and extending to multi-label data.

Contribution

The paper proposes MCDA, a novel LDA variant that emphasizes small between-class distances and extends to multi-label data, overcoming limitations of traditional NLDA.

Findings

MCDA outperforms 10 single-label approaches in accuracy.

MCDA surpasses 4 multi-label approaches in F1 scores.

MCDA is effective on diverse datasets.

Abstract

Linear Discriminant Analysis (LDA) is a widely-used supervised dimensionality reduction method in computer vision and pattern recognition. In null space based LDA (NLDA), a well-known LDA extension, between-class distance is maximized in the null space of the within-class scatter matrix. However, there are some limitations in NLDA. Firstly, for many data sets, null space of within-class scatter matrix does not exist, thus NLDA is not applicable to those datasets. Secondly, NLDA uses arithmetic mean of between-class distances and gives equal consideration to all between-class distances, which makes larger between-class distances can dominate the result and thus limits the performance of NLDA. In this paper, we propose a harmonic mean based Linear Discriminant Analysis, Multi-Class Discriminant Analysis (MCDA), for image classification, which minimizes the reciprocal of weighted harmonic mean of pairwise between-class distance. More importantly, MCDA gives higher priority to maximize small between-class distances. MCDA can be extended to multi-label dimension reduction. Results on 7 single-label data sets and 4 multi-label data sets show that MCDA has consistently better performance than 10 other single-label approaches and 4 other multi-label approaches in terms of classification accuracy, macro and micro average F1 score.