Robust self-healing prediction model for high dimensional data

TL;DR

This paper introduces a robust self-healing hybrid prediction model for high-dimensional data, especially in medical applications, which cleans data errors without discarding data, enhancing accuracy and reliability.

Contribution

It proposes a novel self-healing approach that cleans data errors in high-dimensional datasets, integrating ensemble classifiers and genetic algorithm-based feature selection.

Findings

Outperforms existing high-performing models in accuracy

Effectively handles data errors without data loss

Improves prediction reliability in medical data

Abstract

Owing to the advantages of increased accuracy and the potential to detect unseen patterns, provided by data mining techniques they have been widely incorporated for standard classification problems. They have often been used for high precision disease prediction in the medical field, and several hybrid prediction models capable of achieving high accuracies have been proposed. Though this stands true most of the previous models fail to efficiently address the recurring issue of bad data quality which plagues most high dimensional data, and especially proves troublesome in the highly sensitive medical data. This work proposes a robust self healing (RSH) hybrid prediction model which functions by using the data in its entirety by removing errors and inconsistencies from it rather than discarding any data. Initial processing involves data preparation followed by cleansing or scrubbing through context-dependent attribute correction, which ensures that there is no significant loss of relevant information before the feature selection and prediction phases. An ensemble of heterogeneous classifiers, subjected to local boosting, is utilized to build the prediction model and genetic algorithm based wrapper feature selection technique wrapped on the respective classifiers is employed to select the corresponding optimal set of features, which warrant higher accuracy. The proposed method is compared with some of the existing high performing models and the results are analyzed.