Deep Network Ensemble Learning applied to Image Classification using CNN Trees

TL;DR

This paper introduces a simple, sequential ensemble method using multiple ResNet50 deep networks inspired by decision trees, which outperforms single ResNet50 classifiers on ImageNet and natural images datasets.

Contribution

It presents a novel, efficient ensemble approach combining deep networks with decision tree concepts, addressing complexity and tuning issues of traditional ensemble methods.

Findings

Outperforms single ResNet50 on ImageNet dataset

Effective ensemble strategy inspired by decision trees

Code available for reproducibility

Abstract

Traditional machine learning approaches may fail to perform satisfactorily when dealing with complex data. In this context, the importance of data mining evolves w.r.t. building an efficient knowledge discovery and mining framework. Ensemble learning is aimed at integration of fusion, modeling and mining of data into a unified model. However, traditional ensemble learning methods are complex and have optimization or tuning problems. In this paper, we propose a simple, sequential, efficient, ensemble learning approach using multiple deep networks. The deep network used in the ensembles is ResNet50. The model draws inspiration from binary decision/classification trees. The proposed approach is compared against the baseline viz. the single classifier approach i.e. using a single multiclass ResNet50 on the ImageNet and Natural Images datasets. Our approach outperforms the baseline on all experiments on the ImageNet dataset. Code is available in https://github.com/mueedhafiz1982/CNNTreeEnsemble.git