Deep N-ary Error Correcting Output Codes

TL;DR

This paper introduces deep N-ary ECOC, a novel ensemble method that decomposes multi-class problems into simpler tasks using deep neural networks, with new parameter sharing architectures to improve training efficiency and performance.

Contribution

It proposes three parameter sharing variants for deep N-ary ECOC, enabling effective training with deep neural networks and demonstrating superior performance on image and text classification tasks.

Findings

Deep N-ary ECOC outperforms other ensemble methods.

Parameter sharing architectures improve training efficiency.

Effective across various neural network backbones.

Abstract

Ensemble learning consistently improves the performance of multi-class classification through aggregating a series of base classifiers. To this end, data-independent ensemble methods like Error Correcting Output Codes (ECOC) attract increasing attention due to its easiness of implementation and parallelization. Specifically, traditional ECOCs and its general extension N-ary ECOC decompose the original multi-class classification problem into a series of independent simpler classification subproblems. Unfortunately, integrating ECOCs, especially N-ary ECOC with deep neural networks, termed as deep N-ary ECOC, is not straightforward and yet fully exploited in the literature, due to the high expense of training base learners. To facilitate the training of N-ary ECOC with deep learning base learners, we further propose three different variants of parameter sharing architectures for deep N-ary ECOC. To verify the generalization ability of deep N-ary ECOC, we conduct experiments by varying the backbone with different deep neural network architectures for both image and text classification tasks. Furthermore, extensive ablation studies on deep N-ary ECOC show its superior performance over other deep data-independent ensemble methods.