Dimension Reduction by Mutual Information Discriminant Analysis

TL;DR

This paper introduces MIDA, a new discriminant analysis algorithm that uses one-dimensional mutual information estimations for effective feature extraction in high-dimensional data, demonstrating robust performance across various datasets.

Contribution

The paper proposes a novel discriminant analysis method based on one-dimensional mutual information estimations, improving feature extraction in high-dimensional spaces.

Findings

MIDA outperforms or matches existing algorithms across multiple datasets.

MIDA is robust to different data characteristics.

Efficient estimation of mutual information enhances high-dimensional data analysis.

Abstract

In the past few decades, researchers have proposed many discriminant analysis (DA) algorithms for the study of high-dimensional data in a variety of problems. Most DA algorithms for feature extraction are based on transformations that simultaneously maximize the between-class scatter and minimize the withinclass scatter matrices. This paper presents a novel DA algorithm for feature extraction using mutual information (MI). However, it is not always easy to obtain an accurate estimation for high-dimensional MI. In this paper, we propose an efficient method for feature extraction that is based on one-dimensional MI estimations. We will refer to this algorithm as mutual information discriminant analysis (MIDA). The performance of this proposed method was evaluated using UCI databases. The results indicate that MIDA provides robust performance over different data sets with different characteristics and that MIDA always performs better than, or at least comparable to, the best performing algorithms.