Missing Data Imputation for Classification Problems

TL;DR

This paper introduces a new iterative kNN imputation method that uses class weighted grey distance and mutual information to improve missing data imputation for classification tasks, especially with heterogeneous data.

Contribution

It proposes a novel imputation technique combining class weighted grey distance and mutual information, enhancing classification accuracy with missing data.

Findings

Outperforms existing kNN, MICE, and missForest imputation methods.

Improves classification performance on simulated and UCI datasets.

Effective with various missing data rates.

Abstract

Imputation of missing data is a common application in various classification problems where the feature training matrix has missingness. A widely used solution to this imputation problem is based on the lazy learning technique, $k$-nearest neighbor (kNN) approach. However, most of the previous work on missing data does not take into account the presence of the class label in the classification problem. Also, existing kNN imputation methods use variants of Minkowski distance as a measure of distance, which does not work well with heterogeneous data. In this paper, we propose a novel iterative kNN imputation technique based on class weighted grey distance between the missing datum and all the training data. Grey distance works well in heterogeneous data with missing instances. The distance is weighted by Mutual Information (MI) which is a measure of feature relevance between the features and the class label. This ensures that the imputation of the training data is directed towards improving classification performance. This class weighted grey kNN imputation algorithm demonstrates improved performance when compared to other kNN imputation algorithms, as well as standard imputation algorithms such as MICE and missForest, in imputation and classification problems. These problems are based on simulated scenarios and UCI datasets with various rates of missingness.