Deep Convolutional Neural Network Ensembles using ECOC

TL;DR

This paper explores the use of error correcting output coding (ECOC) as an ensemble method for deep neural networks, aiming to improve accuracy while managing training complexity, and introduces new design strategies and a combinatory technique.

Contribution

It proposes novel ECOC-based ensemble strategies for deep networks, addressing the accuracy-complexity trade-off and demonstrating superior performance over existing ensemble methods.

Findings

ECOC ensembles outperform traditional methods like averaging and gradient boosting.

The proposed combinatory technique achieves the highest classification accuracy.

Extensive comparative study validates the effectiveness of ECOC designs.

Abstract

Deep neural networks have enhanced the performance of decision making systems in many applications including image understanding, and further gains can be achieved by constructing ensembles. However, designing an ensemble of deep networks is often not very beneficial since the time needed to train the networks is very high or the performance gain obtained is not very significant. In this paper, we analyse error correcting output coding (ECOC) framework to be used as an ensemble technique for deep networks and propose different design strategies to address the accuracy-complexity trade-off. We carry out an extensive comparative study between the introduced ECOC designs and the state-of-the-art ensemble techniques such as ensemble averaging and gradient boosting decision trees. Furthermore, we propose a combinatory technique which is shown to achieve the highest classification performance amongst all.