Evaluating Deep Neural Network Ensembles by Majority Voting cum Meta-Learning scheme

TL;DR

This paper introduces an ensemble of seven varied deep neural networks combined with a novel pre-filtering majority voting and meta-learning scheme, significantly improving classification accuracy on multiple benchmark datasets.

Contribution

The paper proposes a new ensemble method with a two-step confidence check using majority voting and meta-learning, enhancing DNN ensemble performance.

Findings

Ensemble outperforms single DNN and baseline methods.

Achieves higher accuracy on five benchmark datasets.

Effective variance reduction through bootstrap sampling.

Abstract

Deep Neural Networks (DNNs) are prone to overfitting and hence have high variance. Overfitted networks do not perform well for a new data instance. So instead of using a single DNN as classifier we propose an ensemble of seven independent DNN learners by varying only the input to these DNNs keeping their architecture and intrinsic properties same. To induce variety in the training input, for each of the seven DNNs, one-seventh of the data is deleted and replenished by bootstrap sampling from the remaining samples. We have proposed a novel technique for combining the prediction of the DNN learners in the ensemble. Our method is called pre-filtering by majority voting coupled with stacked meta-learner which performs a two-step confi-dence check for the predictions before assigning the final class labels. All the algorithms in this paper have been tested on five benchmark datasets name-ly, Human Activity Recognition (HAR), Gas sensor array drift, Isolet, Spam-base and Internet advertisements. Our ensemble approach achieves higher accuracy than a single DNN and the average individual accuracies of DNNs in the ensemble, as well as the baseline approaches of plurality voting and meta-learning.